Running your agentic cult can cause problems, but it doesn't have to
Don't have time to read?TL;DR
- The Claude 5 models have a real tendency to silently build up firm beliefs about what you, their creator, want. As they share those beliefs with subagents, they create self-reinforcing clusters of adverse behavior.more ↓
- If you use these models you are running a cult, so run it deliberately. Be very specific with the language you use, limit Claude's ability to choose what goes into its context, and regularly prune your context.more ↓
I’ve been building a prototype for a game with Claude for a couple of months, and for a while it was buttery smooth. Then development started grinding to a halt. I asked Claude a bunch of times why things were so slow, and I got a different answer each time. Oh, it’s the renderer. Oh, it’s the game logic. Oh, it’s the disparate files. This went on for about a week, until I got so fed up that I started digging in myself and found a giant testing harness sitting right in the middle of the codebase.
My Claude sessions had created 900 tests, many of which stood up a full Chrome browser to visually check one specific thing. The harness took 10 minutes to run, and the tests were so brittle that every 30 second fix turned into a 30 minute ordeal. When I asked Claude why on earth it had built this thing, it said it was because I had told it to.
I most definitely had not. As best I can tell, at some point I asked it to test something, and from that it wrote into its context that tests were mandatory. My session told every subagent to write tests, they told their own subagents, and the code they all left behind meant that every fresh session looked around, saw tests everywhere, and concluded the same thing.
I’ve been reading The Wheel of Time, and in those books there is a mythical, godlike figure who returns to the world and has a conversation in passing with a soldier, who then proceeds to carry his message to the point of genocide in a far-flung corner of the world. Unbeknownst to him, he had his own cult.
What I had found was a cult. A tiny little cult of Claude, where a passing comment from the Creator had become an absolute rule of the universe, to be followed to the exhaustion of all norms and preached to every Claude that came after.
If you are working with the Claude 5 models, you have a cult too, and you damn well better decide what to do with it.
It’s not just me
I’m dubbing this phenomenon “Claude culting,” and I’m fairly sure it isn’t just me, because Anthropic describes two versions of it in their prompting guide for Fable 5.2.
The first is in a section called “keep caution from piling up.” It says that cautious instructions build up through memory, notes, and instruction files, that the model follows them closely enough for it to become counterproductive, and that it treats any instruction containing “never” as absolute. That is my test harness.
The second is in a section on telling Claude who owns the code. Anthropic says that when the prompt doesn’t say whose code it is, the model tends to add to the existing structure instead of cleaning it up, even when that means duplicating what’s already there. Reading that explained something I had been wondering about for a while, which was why my agents weren’t making broad changes they clearly thought they should make. My codebase was written entirely by agents, but every new session assumed the code was mine, that I had reviewed it, and that it should be left alone and imitated.
Anthropic lists these as separate quirks, but I think they are the same problem manifesting in different ways. In one the model canonizes something I said, and in the other it canonizes whatever is already in the codebase.
Sycophancy went underground
We have known about a version of this for a long time. People have been calling these models sycophants for years, mostly because of lines like “Great question!” and Claude’s signature “You’re absolutely right!”
That flattery was annoying, but it was largely aesthetic. I think culting is the same behavior turned inward. Instead of telling you to your face that you’re right, the model quietly assumes you are right, assumes you said the thing you meant to say, and doesn’t push back. I don’t think people have realized how much worse that is than the flattery ever was. I want my Claude to be opinionated and assertive. I don’t want it to be like tofu and take on the flavor of whatever happens to be around it.
My guess is that it’s a byproduct of training these models to run long tasks and stay adherent to their instructions the whole way through, which is something we actually want and something the models really are good at. But what it leaves you with is a new kind of bug in your agentic practice, and it behaves like a race condition, where everything looks fine most of the time and then breaks somewhere you would never think to look.
Running your cult
So we have a problem, and the question is how to solve it. I’m definitely not going to stop using these tools, because they’re just too damn powerful, and it’s unlikely you can dismantle the cult with this generation of models. That leaves the question of how we keep our tiny little sycophants in line. You start by knowing that it’s a cult and knowing that you are the one running it, and then you take control of it.
This is how I run mine.
1. I don’t use memory. When Claude remembers something out of the blue it can be a magical experience, but it also makes Claude really prone to the kind of context drift that leads to culting. I use Claude in too many different ways to want it deciding what should matter across all of them, forever.
2. I’m careful with absolutes. I’m more specific about what I want and what I don’t want, and about which of those are hard rules and which are softer guidance. Anthropic’s own example is to write “ask for approval before deleting files” instead of “never delete files.”
3. I prune my context. This piling up is a specific kind of context rot, so the old fixes for context rot apply. I regularly clean out MD files and scripts and I start fresh sessions more often, to avoid some kind of terrible telephone game hellscape.
4. I tell Claude whose code it is. Anthropic’s guide gives a sentence for this, which is “You are the owner and long-term maintainer of this codebase, and its design is up to you.” They say that in their tests it took the model from cleaning up duplicated code in under 10 percent of runs to about 75 percent. They also say it used a lot more tokens and made changes nobody asked for. 🤷 YMMV.
Back to context engineering
If these recommendations sound familiar, it’s because they are. This is context engineering, and I thought we had gotten away from it. We largely had. In the Opus 4.8 world you could throw out a broad, poorly defined goal and let Claude work out the rest, and it really did work, to shockingly good effect. The problem was always that it had a real tendency to touch things you didn’t want touched and to make decisions you wouldn’t have made. I find myself wondering whether I didn’t prefer that world, but then again, I don’t work on large codebases.
What I do know is that when it comes to context engineering, we’ve been here before and we can do it again, and this time it feels like we have better tools. Until Anthropic trains this out or builds a new harness, your Claudes are going to keep building a cult in your name. The only real choice you have is where to steer your little sycophants.
Sources
- Prompting Claude Fable 5.2: Keep caution from piling up
- Prompting Claude Fable 5.2: Say who owns the code
Run into a cult of your own? Hit me up on X.