Guardrail Engineering: The New Role of Tests and Specs on an AI Production Line

Preface

The first post defined guardrails as “tools for widening the neck” — hand the checks a person would otherwise eyeball every time to a machine, and keep human acceptance bandwidth for what machines can’t verify. The last post saw the flip side of that in a green PR: all 1,294 tests passed, but every one of them used a single stash, exactly the point where the wrong answer and the right one coincide. A test that can’t tell wrong from right isn’t a guardrail; it’s decoration.

So this post doesn’t argue about whether to write tests — when AI writes your tests for you, the cost side of that question has already changed. It asks two questions you can measure, using GitCrisp as the sample:

1. What has it stopped? — five months of CI red runs.
2. What would it miss? — break the code on purpose and see whether the tests go red.

I had an AI dig up and run both ledgers: one reads the GitHub Actions run history, the other is a mutation-testing run.

Ledger one: what five months of CI stopped

GitCrisp’s CI started on May 17 and had run 231 times by October 5: every PR gets a full pytest run on each of Linux, Windows and macOS, plus ruff and mypy (mypy only checks the domain and application layers).

The numbers themselves are unremarkable: 15 red runs, half of them tests, a few each from lint and infrastructure, and mypy never red once. What’s worth telling is two groups among those 8 test failures.

The pygit2 1.20 silent bug. Dependabot opened a PR bumping pygit2, and CI went red on all three platforms — three runs in a row, blocked for eleven days. pygit2 1.20 had removed the old GIT_* constants, and the code GitCrisp uses to read repo state looked them up with getattr(pygit2, name, None) — so the missing constants raised nothing; the lookup table just went empty, and state() fell all the way through to its default, CLEAN. The commit that fixed it says so plainly: “The user-visible effect is worse than a failed import would have been.” A repo stuck in the middle of a merge conflict would show “no operation in progress”; the conflict banner would never appear, and its abort/continue actions would be unreachable.

Anyone who read the incident post will recognise the shape: failure as silence. It doesn’t crash, doesn’t throw — it just quietly tells you everything is fine. That’s the hardest kind of bug to notice in Production, and here it was stopped at the PR by a few tests that hard-code “mid-merge, the state must read MERGING” — red on all three platforms.

Bugs that only show up on one operating system. A layout overlap only on macOS (off by 1px), a heap corruption only on Windows; and two more runs that went red only on Linux or only on Windows, whose logs have since expired, so there’s no telling which test. The three-platform matrix makes CI take three times as long, and these are what it buys — on a single-platform CI, every one of them would have been green.

One more entry, for honesty: the only PR merged on a red run was red because of a flaky test — a debounce test racing the scheduler, unrelated to that PR’s change. A later PR fixed it. The real cost of a flaky test isn’t that one merge; it’s that it teaches people “sometimes you can ignore red”.

Ledger two: break it on purpose — does anything go red?

Ledger one answers “what has it stopped”, but it only sees the times someone actually got something wrong. To know how strong a guardrail is, you have to ask the reverse: if the code were wrong, would it go red? That’s what mutation testing does — deliberately change one thing in the code (== to !=, and to or, a number plus one, a return value replaced by None), run the full test suite, and see whether any test goes red. If one does, that mutant is “killed”; if everything stays green, it “survived” — an error the test net didn’t catch.

I had an AI randomly pick 15 spots in each of GitCrisp’s four layers (domain only had 4 mutable spots, since that layer is almost all data structures and interface definitions) — 49 mutants in all, each run against the full 1,294 tests. The 24 survivors were then classified one by one: 7 were equivalent mutants — changes that don’t affect behaviour, like a default argument on a Protocol interface (which no implementation ever executes) or a layout margin going from 4px to 5px. It’s correct for tests to miss those; they don’t count as gaps.

With equivalent mutants excluded, the tests killed 25 of 42 effective mutants — about six in ten. The number only means something layer by layer:

The application layer is the strongest, at 80%. It’s almost all one-line pass-throughs — return self._reader.get_stashes() — and most of them have a dedicated delegation test: “call the query, check it hands its arguments to the reader untouched.” Those tests look dumb, but they’re exactly checking “does it go red when it’s wrong”.

Infrastructure and presentation both come in under half. Several of the 17 real gaps among the survivors are very concrete: flip the == that matches the file name in a diff to !=, and you get another file’s hunks back; flip the != that updates the cache after deleting a remote tag to ==, and the cache ends up holding only the tag you just deleted; invert the check for a locked worktree, and the wrong error dialog pops up. All tests green.

The most telling part is the seams. Three application-layer mutants survived, including the “file history” query returning None outright — the entire file-history feature broken, and not one of the 1,294 tests went red. Not because nobody tests file history: infrastructure has tests for its reader, and presentation tests that clicking the menu emits the file-history signal. The problem is that on the presentation side the queries are a MagicMock, and on the infrastructure side the tests call the reader directly — each side is right on its own, and when the one-line pass-through between them breaks, no layer’s tests can see it. The only thing guarding the space between layers is the e2e journeys: 1.7% of all the tests, and they killed 4 of the 25 mutants that died.

The guardrail recursion and its stop-loss

The first post left a question half-answered: the tests are AI-written too, so who accepts the tests? That recursion has no bottom — whatever accepts the tests needs accepting too — and the stop-loss offered then was “a person spot-checks the guardrails themselves”.

This experiment makes that spot-check concrete: don’t read the tests line by line; break the code and see whether the tests notice. Nobody can read twenty thousand lines of tests, but 49 mutants on six parallel workers, under an hour of machine time in total, drew the map above — which layer is strong, which is weak, and what the errors that slip through look like. It’s a view a person can never get from review: review sees what the tests say; mutation sees what the tests stop.

It’s the other direction of the last post’s bug-planting experiment: that one was “plant bugs, see if the reviewer catches them”, this one is “plant bugs, see if the tests catch them”. Both ask the same question — is the line of defence you think you have actually there?

The return on guardrails

How do you work out the return on guardrails? The honest answer: a bug that was stopped has no price tag — if that pygit2 silent bug had shipped in a release, there’s no number for how much of users’ time and trust it would have cost. But the cost structure on both sides is visible:

• The cost side has dropped sharply in the AI era. GitCrisp has almost as many lines of tests as lines of code (20.3k against 19.8k), a heavy ratio back when people wrote tests by hand; but 528 of this project’s 904 non-merge commits are marked as co-authored by Claude, so the marginal cost of writing a test is now close to the marginal cost of writing the code. The costs still borne by people are two others: CI time (the three-platform matrix triples every run) and the trust cost of flaky tests.
• The return side hasn’t changed shape. It’s still sparse and unpredictable — 15 red runs in five months, of which maybe one or two really mattered. But those one or two are the silent bug and the bug that only shows on one platform — precisely the two kinds a person is least likely to catch in review.

So on an AI production line, the question moves from “should we write tests” to “are the tests strong enough”. AI has driven down the cost of the first; not the second, because tests written by AI will just as happily use a single stash.

Reflection

Test line count is output, not strength

GitCrisp has twenty thousand lines of tests, and that number is easy to hold up as proof that “the guardrails are thorough”. This experiment says it isn’t: eight hundred lines of tests in the application layer buy an 80% kill rate; thirteen thousand seven hundred lines in presentation buy 45%. Line count measures how many tests the AI wrote, not how many errors the tests stop. Once a metric is easy for an AI to push up, it stops telling you how strong your guardrails are — line count and coverage both. A mutation kill rate is harder to inflate, because to raise it, the tests actually have to go red when the code is wrong.

The errors that slip through live in the seams

Clean Architecture makes every layer easy to test. That’s its strength, and it’s also the blind spot this experiment found: each layer’s tests stay inside their own box, and the line between boxes belongs to no layer. That’s exactly where an AI production line is most likely to break — AI is very good at getting each layer right on its own and backing it with tests, but “does the next layer actually use what this layer hands it?” falls inside no layer’s tests. Which is why e2e tests deserve a fresh valuation on an AI production line: they’re slow, few and hard to write, but they’re the only guardrail that crosses the seams.

Guardrails need accepting too

One idea keeps coming back in this series: a neck doesn’t exist because you drew it; it exists because, time after time, it actually stops things. Guardrails are the same — a test suite that has never been deliberately broken might be a guardrail or might be decoration, and you can’t tell which. The CI ledger can only tell you what it has stopped; mutation tells you what it will miss. A guardrail isn’t something you finish writing. It needs breaking and accepting on a schedule, the same way it accepts the code.

Earlier in this series: #1 the responsibility funnel · #2 a clock on the neck · #3 the neck on the board · #4 the responsibility premium · #5 spec as code · #6 the craft of acceptance