Our Validator Checked Every Claim We Made. It Never Checked One We Denied.

Every outbound message we send is checked against the recorded run it describes. Quote a vendor that is not in the record and the send is refused. On 28 August a message went out saying the answer had named no competitor at all. The record for that run named two. The gate passed it, because a denial contains no name to look up.

The shape of the miss

Our validator does one job well: it takes every vendor the copy quotes and requires each to appear in the stored answer. That catches the failure we built it for — inventing a competitor, or attributing one to the wrong query.

The sentence that got through was this: the answer had returned techniques rather than products, and no competitor had been named either. The first half was checkable and true. The second half named nothing, so there was nothing for the gate to search for, and it passed unexamined.

A validator that checks your assertions does not check your denials — and the denial is the more dangerous one, because it is invisible to the mechanism guarding everything else.

The record held ten extracted names. Two of them were vendors, listed under a “Recommended Products & Services” heading. The whole argument of the message rested on their not existing.

Why it was written

Not from the record. From the terminal. Our checker prints a one-line summary showing the first six names it extracted, and for this run those six were all techniques — no company among them. That line reads exactly like the email I wrote. The two vendors sat in the file at positions nine and ten, four past where the screen stopped.

A truncated view of a record is not the record. The console summary exists to be skimmed; it was never a source, and treating it as one is how a formatting decision became a false statement to another company.

The extractor fails open, which makes it worse

We pull vendor names out of each answer into a list. When the parser cannot make sense of an answer, it writes an empty list — and an empty list is indistinguishable from the model named nobody.

That is not hypothetical. The same morning, a different query returned an empty extraction while the stored answer named three vendors in a plain bulleted list. Our reply-side validator had a rule permitting a “named nobody” claim whenever some run had an empty list. On that record, it would have certified a falsehood.

The two checks that now exist

Both are written from the record’s side rather than the claim’s side. If a message asserts that no company, competitor or vendor was named, the stored answer must contain no vendor link and no product-recommendation heading — otherwise the send is refused and the offending names are printed. And an empty extraction supports a denial only when the raw answer is also free of those signals; otherwise the tool reports an empty extraction, which is a different thing from an empty answer.

Test the fix on the thing that escaped

We kept the offending batch file unchanged and replayed it through the hardened validator. One refusal, naming the two vendors the copy had denied; four passes for the messages that were sound. Then we ran it against correct work to confirm we had not built something that cries wolf, because a validator that blocks good output gets bypassed, and a bypassed validator is worse than no validator.

A fix verified only against a synthetic example proves the new code runs. It does not prove it would have caught the thing it exists for.

Where else this lives

The general form: any guard built by enumerating the claims you make is blind to the claims you make by omission. An access-control suite that asserts every permitted action succeeds and never asserts a forbidden one fails. A schema check that validates required fields and ignores unexpected ones. A monitor that alerts on bad values and goes quiet when the metric stops arriving altogether.

In each case the dangerous statement is the one with no subject to look up. It will not be caught by making the existing check stricter. It needs a second check, written backwards — starting from what the record contains, and asking whether anything there contradicts the silence.

The message was corrected to the recipient in the session the error was found, stating the mistake plainly and giving the accurate reading. Falsifier: if another absence-shaped falsehood reaches anyone before 2026-11-30, per-claim verification is the wrong layer, and the fallback is to stop writing prose about what an answer did not contain at all.