Our Emails Offered Data Our Checker Had Thrown Away

Every visibility pitch our outbound engine has sent ends by offering the raw data behind it: all six AI answers, with what the model said about each company it named. For two months, the checker that produced those answers kept one of them, cut short, and threw the rest away. We found out when the first prospect accepted the offer.

What the pitch promised

The pitch is built on a measurement. We ask an AI model, through the Gemini API, the question a buyer in the prospect’s category would ask, several times over, because a single answer from a model is one draw from a distribution, not a fact. If the prospect is absent from every run, the email says so, quotes the question, lists who was named instead, and closes with:

I have all six answers, with what it said about each name. Want them?

What the record held

The checker’s output row looked complete. It was not:

"named_examples": results[0]["named"],
"answer_excerpt": results[0]["answer_excerpt"],   # run 1, ans[:900]

Run 1 was kept, truncated to 900 characters. Runs 2 and 3 were each reduced to one boolean, whether the brand appeared, and the text was discarded. Across two questions, the email offered six answers and the record could supply two partial ones.

A claim nothing stores is not a lie yet. It is a lie waiting for someone to accept the offer.

Why it survived for months

Over 200 first touches went out before anyone asked for the answers. Until someone did, the delivery path did not exist as running code, only as a sentence in a template. An unexercised path is not a working path; it is an untested one, and the test is run by a customer.

We do check pitch claims before they send. A draft checker compares what an email says the model said against the stored runs, and blocks a draft that contradicts them. That kind of checker has blind spots of its own; we have written about one that checked every claim an email made and none it denied. It could not see this one either. “I have the answers” is not a claim about the model at all. It is a claim about our filing cabinet, and nothing compared the cabinet to the promise.

The rule: offer it, store it

The checker now writes every run in full, with no truncation, into a runs_detail field alongside the exact prompt it sent. The old run-1 fields stay for compatibility. Rows written before the change have no full text and cannot be delivered as promised; for those, the instruction in the code is to re-run the stored prompt rather than quote what is not there.

The general form is a check worth running on any outbound template: for every artifact the message offers, name the field that holds it. If you cannot point at the field, the offer is not yet true.

Telling the prospect

The reply did not quietly re-run the check and present the result as if it had been there all along. It opened with the correction, in one sentence: “I offered six; our checker had kept only run 1 of each question, truncated. Our bug.” Then it gave both draws in full: the original as it was actually stored, and a fresh one with every word kept. Admitting we had less than we offered cost a sentence. Being caught handing over a padded file would have cost the conversation.

The ceiling that nearly blocked the delivery

Our mailer refuses any email body over 200 words. Six full model answers are several times that, so the one email a prospect had explicitly asked for would have been refused by our own gate.

The tempting fix was to exempt a marked appendix from the word count. That was proposed and refused as a weakening of a guard, and the refusal was right: the ceiling exists to keep pitches short, and a carve-out is how a ceiling stops being one. Instead, the mailer gained the ability to attach a file, sent as an ordinary MIME multipart message with the data in a part marked as an attachment. The body stayed under 200 words, the data rode alongside it, and we confirmed the attachment was present on the sent message itself rather than trusting the send call’s return value.

What to take from it

Audit your templates for promises, not just for facts. A sentence that offers something (“I can send you…”, “the full report is available”, “happy to share the data”) is a contract with your storage layer, and it is the one kind of claim a fact-checker will not flag, because it is not wrong yet.

It is the same shape as an empty result read as a successful request and a gate that printed every verdict and saved none: the visible output looked right, and what sat behind it was never checked. Here the thing that finally checked it was the first person who said yes.

From our own engineering records, October 14, 2026. The storage fix shipped the same day. The prospect is not named, and nothing they wrote is quoted.