Four passes. Eleven days. One number that was right the whole time.

The figure was 31 relationships worth $412K. A checking pass looked at it and said: each of these is capped under $10,000, there are 31 of them, so the ceiling is $310,000. The stated total is $412,880. That's arithmetically impossible on its face.

Clean logic. Confident. Wrong.

The cap was on one column. The total summed a different one. Non-nested, so the per-item ceiling never bounded the total at all. One account in that set carried $5,000 against $268,410.

But the correction had already gone out. Then a later pass reversed it. Then reversed it back. By day eleven a correct entry had been retracted, restored, and retracted again, and the thing being judged was no longer the analysis. It was whether we could be trusted to count.

I went back through a year of corrections after that and sorted them by what caused them. Twenty-three distinct failure modes, every one from a real correction on real work.

Three of the worst were not caught by a verification pass. They were produced by one.

The $412K flip-flop above. A checking agent that "corrected" a right figure to a wrong one by querying the wrong level of the same table, then recommended deleting a live guard on the strength of its own error. A re-verification that rewrote the original query from scratch, quietly dropped one filter, declared the recorded figure unreproducible, and published a wrong number to a customer-facing page.

Same shape all three times. A verification pass's number is a claim, exactly like the thing it was sent to check. It just arrives wearing more authority, so it wins the argument.

That is the whole problem, and I don't think the current answer to it helps.

Ask around about agents producing bad figures right now and you get two answers.

Upstream, you get a semantic layer: put the metric definitions somewhere governed so the model can't invent what "revenue" means. That one is real and I'd take it tomorrow. Dawn Choo put the underlying problem better than I can: "When you tell an LLM to write a query, it will always give you an answer. It will not tell you it guessed at what revenue means, or that it quietly included your internal test accounts, or that it had no idea what to do with refunds. You get a number, no error, and no reason to doubt it."

Downstream, you get a self-correction loop. A guardrail flags an unsupported claim, feeds it back with a targeted correction prompt, the model retries, and the corrected output re-runs through the guardrail.

Read that second one again with the $412K story in your head.

That's the failure, automated. A verifier drawn from the same distribution as the generator inherits its blind spots. Independence of context helps and it is not enough. You have built a machine that produces confident corrections at speed, and confident corrections are the thing that cost me eleven days.

So I built the other one.

numbers-gate reads a finished document, matches each numeric claim against those 23 signatures, and hands back a question plus the specific check that settles it. Then it stops.

It never recomputes anything.

Here's what that looks like. Say a line in your report reads:

Nothing there is arithmetically wrong. The scanner doesn't check the division. What it notices is the shape: a per-unit average with no concentration stated anywhere near it. So it asks.

In the run that produced that signature, one customer worth $18,400 was 24% of the referral bucket, and had arrived through a self-referral URL. Cleaned up, referrals ranked below direct. The average wasn't imprecise. It was reporting one row, and the recommendation built on it pointed at the wrong channel.

The question takes four seconds to ask. Nobody asked it for six weeks.

In: a .md, .html, .txt or .yaml file, optionally your table schema. Out: a list of questions with locations, each carrying the check that answers it. Exit 0 if it only has questions. Exit 1 only for arithmetic impossibility, which is a fact rather than an opinion: a subset cannot exceed its superset.

There's a second half that works the other way round. gates.py is a set of primitives you import into your own pipeline, so a publish fails on a nonzero exit instead of a human noticing later. It covers baseline drift, partition, subset, reconcile-to-zero, concentration, staleness and claim-versus-evidence. Standard library only. No pandas. A gate that can't run in a bare python3 is a gate that gets skipped on the machine where it matters.

It states its own coverage on every clean run. A green check that read a third of a document is worse than no check, because it reads like protection. So a clean scan prints what it covered and names the four modes that still need a human. Those four are the expensive ones.

It refuses to read a PDF. pdftotext -layout splices multi-column pages onto shared lines and destroys the sentence boundaries every detector depends on. Point a linter at a PDF text dump and you get a confident zero on a document it never read. That has already happened here: a two-page deliverable once cleared a "0 violations" run while carrying thirteen. So the scanner declines the PDF and tells you to pass the source.

It shuts up. First sweep over a real repo returned 106 findings, most of them on append-only log rows nobody had touched in weeks. That's a check people turn off. Three narrowings fixed it: only lines added since HEAD, a minimum figure count so prose is skipped entirely, and a <!-- verify-disable: reason --> marker. 106 down to 35, concentrated on that day's actual work. Costs 0.03s when nothing qualifies, 0.21s when something does.

That's the install. No dependencies.

There's a Claude Code Stop hook in hooks/ that runs it on whatever you wrote during a session and reports before the turn ends. Impossibility blocks once. Everything else surfaces as questions.

Out of the box it knows my failure modes, not yours. Four places to change that, roughly in order of how much they'll matter:

hooks/scope.json is the first thing to edit. Point roots at wherever your numbers actually live. Ship it narrow. A hook that scans everything is a hook you'll disable in a week.

RECORD_SURFACES in signatures.py controls mode 12, the one that catches a record claiming something was sent when it never left. It defaults to comms logs and activity files, because "sent" in a deliverable is usually just prose. Set it to whatever your record surfaces are called.

--schema unlocks two modes that are invisible without it. Hand it a JSON map of your tables and columns and it can finally see that email sits on both the transaction table and its parent, which is the whole mechanism behind mode 20. That one hid 512 paid orders and $486,300 from a per-company rollup, concentrated in the large accounts precisely because those have the messiest multi-seat histories. One account read $140 and was actually $4,290.

The SIGNATURES list is just a Python list. Append your own. Each entry is a mode number, a tell, the question, the check that settles it, and optionally a detector function that yields findings. Write the tell first and the regex second. Every detector in there started as a sentence someone had to correct.

Four of the 23 can't be mechanized. They need a person to look, and they're the ones that have cost the most.

The --orphans sweep uses a literal-path heuristic, so it can't see a path built at runtime, and a clean result there is weak evidence. It says so when it runs.

And a clean scan is a floor. Readers treat a green result as a verdict, which is the exact habit mode 16 exists to describe.

One more, and it's the best argument for the whole approach. Two days after I wrote the test suite, I ran it properly for the first time and found that ROOT was assigned but never exported, so three Python blocks inside it raised KeyError and the entire gates.py section had never executed. Not once. The suite failed loudly, which is the only reason I caught it. Had those blocks been written to skip on a missing variable instead of raise, it would have printed green while testing nothing.

That's mode 16. A checker reporting on coverage it doesn't have. Committed by the thing I built to detect it.

Which is more or less the point. The tool that recomputes your numbers will eventually be wrong with authority. The tool that asks you a question can only ever waste four seconds.