Why I Cut Half of the AI Workers I Built

I spent eight months fixing every problem by adding another AI agent. Then I deleted half of them in one weekend, and the system got better. Adding workers had been hiding a design problem, and removing them made me fix it.

For most of this year, my answer to nearly every problem was the same: add another agent. The one-line version of my operator to builder transition is that I stopped adding workers to my system and started fixing the instructions that the existing ones were working from.

I run a personal operating system made of AI agents. They draft, research, check, route and file. For a while, building a new agent felt like progress, because it was the most visible thing I could do and also the easiest.

The Reflex I Did Not Notice

The reflex was simple: whenever something was missing, I put another worker on it. Research felt thin, so I built a second researcher. Summaries ran long, so I added a summarizer. A wrong answer slipped through, so I added a checker, and later I added a checker to watch the checker.

Each addition made sense on its own, which is exactly what made the habit dangerous. No single agent was the mistake. The pattern was the mistake, and a pattern is hard to see when every step in it looks reasonable.

By spring I was running more than a dozen agents. On a diagram it looked like capacity, almost like a real team, and I was a little proud of the count.

The Week It Stopped Working

Then the numbers turned against me. The share of outputs I could ship without rewriting them stopped climbing and started to slide, so I had more agents than ever and less usable work coming out the other end.

When I looked closely, the failures followed a pattern. Two agents would reach different answers on the same task, and I had no rule for which one won. One agent would redo work another had already finished, because neither knew the other had touched it. A weak output would travel all the way down the chain, because each agent trusted whatever the previous one handed it.

What I'd built behaved like a committee rather than a team. A committee doesn't add judgment to the work; it lets everyone assume someone else supplied it.

What More Agents Actually Cost Me

The part I missed for months was the coordination cost. Every agent I added brought more handoffs, more places for a small error to hide and more chances for two confident systems to disagree without telling me, leaving me to notice later.

That cost never showed up inside any single agent. It showed up in the seams between them, which is why adding never felt like the problem: the problem lived in the spaces that adding had created.

A missing agent is a gap anyone can see, while a bad seam between two agents is a gap nobody sees. I kept closing the gaps I could see and kept creating the ones I couldn't.

The Fix Was Subtraction

Three weekends ago I did the opposite of my instinct and cut the agent count by nearly half. I deleted the second researcher, the checker that watched the checker and three others that existed only to patch seams the earlier agents had opened.

The agents I kept, I made precise. Each one got one job and tighter instructions, written in plain language, about what good output looks like. Each decision got a single owner, so no task answered to two agents. And every agent got a clear rule about when to stop and ask me instead of guessing.

That last rule mattered more than any deletion. Most of my failures had come from agents guessing quietly when they should have paused, far more than from agents doing the wrong thing loudly.

Why Precision Beats Headcount

The result was obvious within days. The system got quieter almost at once, with fewer messages passing between agents and fewer duplicate runs. The share of work I could ship without touching it went up in the same week I removed capacity.

This is the counterintuitive part worth keeping: a few sharply briefed workers did better than many vaguely briefed ones. The agents I kept weren't any smarter than before. The design underneath them was clearer, and I'd been treating a design problem as a staffing problem.

Adding hides that confusion. Every new agent lets the builder avoid the harder question, which is why the existing ones keep missing. Subtraction removes the hiding places and pushes the design fault into the open, where it can actually be fixed.

What This Is Really About

This is the operator to builder shift seen up close. The operator answers a gap with more effort and more people, because that's what works on a factory floor when demand spikes, and I did exactly that for fourteen years, mostly with good results.

The builder answers a gap with a better structure, so the same work needs fewer people next time. The operator counts how much is running, and the builder asks how little the system can run and still hit the outcome.

For me, the tell was emotional. Deleting agents felt like going backwards, as if I were admitting the earlier work had been wasted. It hadn't been wasted, because it showed me where the seams were. The pride, though, was an operator's pride, measured in headcount, and I had to let it go to get a better system.

A Concrete Failure

The clearest failure involved my research agent, which gathers sources, and a second verifier agent I'd built whose only job was to catch the first one's mistakes. The theory was clean. In practice both agents worked from the same weak instruction about what counted as a reliable source, so about one time in five they agreed on the same wrong answer, and they agreed with confidence.

So the setup was two agents sharing one blind spot and reporting twice the certainty. The verifier added a stamp of approval and no judgment of its own. A single sharper brief for the research agent, spelling out what a trustworthy source looks like for my work, removed the error the verifier existed to catch, and I deleted the verifier the same afternoon.

Four Questions Before I Build an Agent Now

I turned the lesson into a short gate. Before I build any new agent, I answer four questions honestly:

One. Is the work actually missing, or is an existing agent doing it badly because its brief is vague? This shows whether I have a capacity problem or an instruction problem.

Two. If I add this agent, who owns the decision it touches, and does anything I already run own that same decision? This catches two agents answering to one task before it happens.

Three. What will this agent do when it's unsure, and have I written that rule down? This is the rule that stopped most of my quiet failures.

Four. Could a sharper instruction to an existing agent do this job instead? If the honest answer is yes, I don't build, and that one rule has kept more agents out of existence than anything else I do.

The Principle

Adding workers hides a design problem, and removing them brings it into view. That holds whether the workers are agents, employees, tools or meetings.

The next time something breaks in a system you run, try to resist the reflex to add. Ask whether the existing worker is failing because it's overloaded or because its instructions were never made precise. Most of the time it's the second, and no amount of adding will fix that.

I used to judge my system by how much it could do, and now I judge it by how little it needs in order to do the job. If you build or run anything automated, where did adding more turn out to be the expensive way to avoid fixing one thing?

Pick a time that suits you.

Book a call