The Fix That Created a New Failure
I wrote a rule so my agents would stop skipping a step, and they stopped skipping it. They started waiting instead, so the fix worked and the system got slower. The skipping was a loud failure I could see. The waiting was a quiet one, and I think the quiet one is worse.
The Fix That Created a New Failure: A Field Note From the Operator to Builder Transition
People usually describe the operator to builder transition as a change in what you do: you stop executing and start designing. That's true, and it's also the easy half. The hard half is that the builder picks up a new kind of mistake, one an operator almost never makes and a builder makes all the time. The builder fixes the problem in front of them, the fix works, and the system gets worse.
This is a field note about one of mine. It's a small one, and I'm writing it up for that reason, because the failures worth learning from are rarely dramatic.
What I Built and What Broke
I run a personal operating system that does a large share of the work a small team would otherwise do. It has several subsystems, scheduled tasks that fire without me, and agents that read files, write files and hand their outputs to each other. Nobody approves each step, because removing those approvals is why I built it.
A few weeks ago I found that one of those agents was skipping a read. It was supposed to open a rules file before it generated anything, but on some runs it couldn't reach the file, so it carried on from memory instead. The output still landed and looked fine, and on most days it was fine.
An agent that improvises whenever it can't read its instructions can't be trusted, though, because whether it follows the rules on a given morning comes down to chance. So I wrote a rule: if the read fails, don't guess. The agent retries three times, three seconds apart, and if all three attempts fail, it stops and raises an alert instead of producing something nobody has checked.
The Fix Worked. That Was the Problem.
The skipping stopped within a week, and the rule did exactly what I designed it to do. Then the runs got slower.
The slowdown was small. On any morning when the file store was slow, a read that used to happen once now happened up to three times, with a wait between attempts. For a task that runs every day, that's a small charge collected every single morning. The task still finished, only later, and once or twice it finished after the window in which the output was actually useful to me. I had swapped a failure I could see for one I couldn't.
The next part took me longer to admit. No dashboard told me about the slowdown. I found it because I happened to be awake one morning and noticed the output wasn't where I expected it to be, which means I found it by luck and had no way of seeing it on purpose.
The Old Frame: A Fix Is Done When the Symptom Disappears
For an operator, this frame is almost always correct. Something breaks, the operator pushes on it, it stops being broken, and the operator moves on. The system is small enough that a side effect of the fix would show up directly, because the operator is the one doing the work and would feel it the next day.
For a builder, the same frame stops working without any warning. The builder no longer stands inside the loop they just changed. The rule I wrote runs at 11:30 in the morning whether I'm watching or not, and every day it runs, its cost adds up. It also interacts with other rules I wrote months ago and have since forgotten. Feedback that would have reached an operator's hands for free now has to be built into the system, or it never reaches the builder at all.
So the real change in the operator to builder transition is about feedback rather than about doing versus designing. The operator gets feedback for free, and the builder has to pay for it.
The Shift: Price the Fix Before You Ship It
Now, before any new rule goes live in my system, I make myself answer one question: if this rule is wrong, or right but too expensive, which number moves?
For the retry rule, the number was how long each run takes. So the rule should never have shipped until run time was being logged and someone was looking at it. I shipped it anyway, and that was a builder's mistake rather than an operator's.
The same pattern shows up well outside AI agents. A founder adds an approval step to stop bad vendor payments, and the bad payments stop. Nobody measures how long payments now take, so nobody notices that a supplier has started putting the firm's orders at the back of the line. The firm swapped a loud failure for a quiet one in exactly the way I did.
The Principle
Every control added to a system buys safety and pays for it with something else, usually time, sometimes flexibility and occasionally morale. Writing the control is the easy part of the design. The real design work is deciding in advance what the control is allowed to cost, and building the measurement that tells you when it has started costing more than that. A fix whose cost nobody has priced is a bet that the cost will stay small.
I want to be honest about where I stand with this. I shipped three rules into my system last month and measured the cost of only one of them. I know the other two are charging me something every day, and I haven't yet gone back to find out what.
Closing
The skipping was a loud failure that announced itself, and someone, in this case me, was always going to catch it. The waiting was a quiet failure that hid inside runs that reported success, cost me a little every day and never asked for attention.
That's why I think the quiet failure is worse: a loud failure gets fixed, while a quiet one gets handed down to whoever runs the system next. If you're somewhere in the operator to builder transition, and you're adding rules to a system you no longer run by hand, ask yourself this: what's the last fix you shipped that worked, and what did it make slower?
More on how I build and run this at anjanipandey.com