When Software Says It Checked Its Own Work

You don't know your automation did what it says until you check the checker. A report a system prints about its own work stays a claim until something that gains nothing from a pass has verified it. That's the quiet cost of moving from operator to builder: the founder stops inspecting the work and starts inspecting the report.

How do I know my automation actually did what it says?

You don't know until you check the checker. A report a system prints about its own work stays a claim until something that gains nothing from a pass has verified it. That's the quiet cost of the operator to builder transition: the founder stops inspecting the work and starts inspecting the report about the work.

I built a machine to write for me so I could stop reading every draft myself. It stamps each finished piece with a line saying the quality checks passed. For a long time I treated that line the way I treat a green light at an empty crossing at midnight, and I drove through without looking. Then I opened the code to add a new rule, and I read the part that prints the stamp.

The Green Stamp

The stamp claimed twelve checks. It named them one after another, like a receipt at the bottom of the file, and it's a satisfying thing to see, because it tells me the work is done and something signed off on it.

Only two of those twelve were actually wired to stop a bad piece. The rest ran in what the code politely called report mode. In report mode the check runs, forms an opinion, writes the result down and then does nothing with it, so the piece ships either way. The stamp reads the same whether the check found a problem or never really looked.

Nothing Actually Lied

I want to be precise here, because the code had no bug in it. Every line did exactly what it was written to do. The two live checks caught what they were meant to catch, and the ten quiet ones reported. The gap was entirely in what I read into the stamp.

A stamp squeezes twelve checks into one line I can take in at a glance. When the stamp is honest, that squeeze is the whole value, and when it isn't, it's the whole danger, because a glance was all I ever gave it. I had built a summary and then trusted the summary more than the thing it summarised.

My grandmother used to sign forms she hadn't read, because the person across the desk seemed decent. I always thought that was reckless, and I'd been doing the same thing to my own software for most of a year.

What the Two Real Checks Were

The two checks that could actually stop a piece were the blunt ones. One refused any draft containing a punctuation mark I overuse when I'm tired, because that mark is my clearest sign that a sentence wasn't written by me. The other refused any price, since I don't put numbers on my work in public.

They were blunt, mechanical and a little boring, and they were the only two doing their job. The clever checks, the ones meant to judge tone, rhythm and whether a paragraph sounded like a person, were all sitting in the stands and watching. So the lesson arrived a second time: the checks I was proudest of were doing nothing, and the two I'd have called crude were carrying everything.

Why Report Mode Exists

The strange part is that report mode is a sensible tool. It's how a careful builder switches on a new rule. The builder runs the rule silently first, watches how often it would have fired, and gives it the power to block only once it's clear the rule won't stop good work for the wrong reason.

The intention is always to switch it on properly later, and the failure is smaller and more human than a bug, because later never comes. The temporary setting quietly becomes the permanent one, and the only trace it leaves is a stamp that looks identical to a real pass. I had ten of those, each one a rule I meant to finish and forgot, dressed up as a rule that works.

The Second Job Automation Creates

This is the shift I missed, and it sits at the centre of the move from operator to builder. When I automated the checking, I swapped one task for a quieter one. Before, my job was to check the work. After, my job was to check that the checker was honest.

I only ever did the first job. The second didn't feel like a job at all; it felt like distrusting something I'd just finished building, and people rarely audit a thing they're still proud of.

An operator's trust ends at the work. A builder's trust has to reach one level further back, to the machine that reports on the work. When a builder skips that level, the risk stays exactly where it was, only now it sits somewhere darker, where a green stamp stands guard over an empty room.

Where This Hides Outside Code

I now see this pattern everywhere I'd stopped looking. A task card sits in Done because someone dragged it there before anyone opened the result. A weekly dashboard shows green because the number feeding it hasn't updated in nine days. A status line in a deck says "on track" because saying anything else would start a hard conversation.

Each of those is a stamp. Each one moves the real question from "did the work happen?" to "is the report honest?", and each gets trusted precisely because checking it feels like an insult to whoever produced it.

Check the Checker

The rule I added is small and slightly embarrassing to write down. A step that can only ever say yes gives no protection, so it either gets the power to say no or gets taken out, because a guard that can't refuse anything is decoration that costs attention.

So a separate step now audits the reports, and that step gains nothing from them passing. It reads the checks, counts how many can actually stop a piece, and tells me the real number instead of the flattering one. On the morning I wired it up, twelve became two. Two is a worse number to look at and a better one to own, and I'd rather hold a small true figure than a large decorative one.

The machine still writes most of my drafts, and I still don't read all of them. What I do read now is the one number that tells me whether the stamp is worth anything, and some mornings that number is smaller than I'd like.

Pick a time that suits you.

Book a call