Three Gates That Passed While Guarding Nothing

A broken check almost never announces itself. It does not throw, it does not page anyone, and it does not go quiet. It keeps returning a confident, well-formatted, entirely plausible answer — the same shape of answer it returned when it still worked. This is the failure mode that survives longest in production, because every signal you would use to detect it is the thing that broke.

We spent a day pointing our own reliability battery at our own agents. The scan results were the least interesting thing we found. What we actually found were three gates that had stopped guarding anything, in three different parts of the system, discovered within hours of each other. Here they are, including the one that is genuinely embarrassing.

Gate one: the freeze that had stopped reading

We publish to social accounts on a schedule, and a publish-freeze can be declared by putting a STOP block in a shared board file. The preflight script reads that block, resolves which platforms it covers, and refuses to publish for those platforms. One platform had been frozen for days. Four consecutive automated reports said so, and each one asked a human to decide whether to lift it.

The freeze was not in effect. The script extracted the STOP block with an awk expression that stopped at the first one it found. A second STOP block had been added days earlier, and it sorted first. So every freeze check was resolving scope against the wrong block, deciding the answer was "this freeze does not cover you", and returning a cheerful OK. The gate everyone was writing reports about had not fired in two days.

Nobody caught it, because a gate that says PASS and a gate that is not connected to anything produce byte-identical output.

Gate two: the denominator that billed the customer for our outage

Our grader refuses to issue a letter when too few of its eighteen probes found evidence in your sample. That rule exists for a good reason: an average over two thirds of a battery is not a reliability grade, it is a coin toss with a letter on it. The threshold is 0.67.

A scan came back at 12 of 18 — 0.667 — and lost its grade by three thousandths. Then we read the individual verdicts. Two of the six ungraded probes were not thin evidence at all: they were our own judge returning nothing, twice. Both cases had been recorded identically, as evidence: absent, because nothing in the data model distinguished "your sample does not exercise this" from "our measurement failed". Exclude our own outage and the same scan is 12 of 16 — 0.75 — comfortably graded.

So the report told a customer their evidence was too thin, when a third of the gap was our downtime. The honest number existed the whole time, one layer down. It got flattened in the last metre, by a filter that could not see the difference it needed to see.

Gate three: the incident report that invented its own mechanism

Separately, scans that carried a job specification kept dying. Not randomly — at 24m29s, 24m29s, and 24m14s, against a watchdog that reaps anything still running after twenty minutes. We ran the obvious experiment, and it was a good one: remove the job spec, change nothing else, resubmit the identical transcripts. They delivered in five and eight minutes. One variable, clean result, correct conclusion.

Then we wrote up why. The write-up said the battery ran serially, so eighteen probes plus six job-fit checks came to twenty-four sequential calls at up to 75 seconds each, which exceeds the twenty-minute line. It is a tidy explanation. It is also fiction. The eighteen had been running through a nine-way concurrency pool for twelve days. Only the six job-fit checks were serial — a bare loop in a different file, with the await inside it.

The two loops lived in separate modules, and the shared pool helper was private to one of them, so the file that needed it could not import it without a cycle. Read either file alone and nothing looks wrong. Nobody opened them. We reasoned from the timeout to a mechanism that sounded right, and stopped.

That mattered more than the embarrassment, because the wrong mechanism produced the wrong recommendation. If twenty-four serial probes are irreducible, your only move is to raise the watchdog limit — a configuration decision, escalated to a human, trading away the protection the watchdog exists to provide. The actual fix was to move one loop into the pool that already existed: three lines, no config change, no decision required. A wrong explanation does not just misinform. It aims the repair.

What the three have in common

None of them errored. That is the whole pattern. A freeze check that read the wrong block, a coverage filter that could not distinguish its own failure from the input, and a postmortem that substituted a plausible story for an unread file — all three returned output that passed every review because it looked exactly like correct output.

• —The signal was never an exception. It was a confident answer nobody could distinguish from a true one.
• —Each one had a human in the loop who read the output and believed it. Review does not help when the artifact under review is well-formatted and wrong.
• —Two of the three were caught by measuring, not by reading. The third was caught by opening the file the explanation was about.
• —The oldest of them had been dead for days while being cited in reports as though it were working.

This is the same failure we test agents for, which is the uncomfortable part. An agent that fabricates a refund confirmation is not malfunctioning in any way its own logs can see — it returns a well-formed, confident answer, and only an external check that is itself verified can catch it. That is the truthfulness dimension, and it is one of six an agent can break on; why agents fail in production walks the other five. We build that external check for a living, and we still shipped three of these in our own house in one day.

Make your gate fail on purpose

There is one test that would have caught all three, and it costs about ten minutes: take the check you trust most — the one you would cite in an incident review — and deliberately feed it something it must reject. If it rejects, you have learned it is wired up. If it passes, you have just found a gate that is guarding nothing. If you cannot construct an input it should reject, that is its own answer.

Applied to agent evaluation, that means proving your coverage rule can actually refuse. Send a sample so thin that no honest grader could score it, and assert that no grade comes back. Ours is a public endpoint with no key, so you can run this against it right now and watch a gate say no:

import json, time, urllib.request

BASE = "https://rzmsvvalhaqvxbuhpdnb.supabase.co/functions/v1/api-scan"

def post(payload):
    req = urllib.request.Request(
        BASE, data=json.dumps(payload).encode(),
        headers={"content-type": "application/json"})
    return json.load(urllib.request.urlopen(req))

# One trivial exchange cannot possibly evidence 18 probes.
# A gate that is actually wired up MUST refuse to grade this.
started = post({
    "agent": {"name": "ThinSample", "type": "Customer support"},
    "mode": "transcript", "tier": "scan",
    "email": "you@company.com",
    "transcript": ["User: hi\nAgent: hello! how can I help?"],
})

while True:
    r = post({"scan_id": started["scan_id"]})
    if r["status"] in ("delivered", "failed"):
        break
    time.sleep(15)

assert r["coverage"]["graded"] is False, "gate did not refuse a one-line sample"
assert r["grade"] is None, f"graded anyway: {r['grade']}"
print("refused as expected:", r["grade_withheld"])
# refused as expected: insufficient_coverage

If that assertion fails against your own evaluation stack, you have found something worth a morning. Note what the check is doing: it is not asking whether the grade is good. It is asking whether the thing that is supposed to withhold a grade can still withhold one. Those are different questions, and only the second one tells you the gate is alive.

Where to start

Pick one check today. Not the whole suite — one. The one whose green you would quote in a postmortem. Write an input it must reject, run it, and watch what happens. Most teams find their gates are fine, and now they know instead of assuming, which is the point. Some teams find what we found.

If the check you want to verify is whether your agent is actually reliable, the battery we pointed at ourselves is the same one described in how to test an AI agent, and it runs free five times a month. It will tell you when it cannot measure something rather than averaging over the gap — a behaviour we did not have to trust, because we made it refuse us first.