Our Email Gate Printed Every Verdict and Saved None of Them

Our outbound email runs through a gate that is supposed to drop bad recipients before anything is sent. For about a month it dropped none. It computed a verdict for every new address, printed that verdict on screen, and then let the send proceed as if no verdict existed. We found it by asking what the gate would do with one specific address, not because anything had gone visibly wrong.

What the gate claims to do

Before a batch goes out, the mailer runs an address verifier over its recipients. Then it drops every recipient whose stored verdict falls in a blocked set:

blocked = {"INVALID", "NO_MX", "ROLE", "NULL_MX", "WRONG_AUDIENCE"}

The first four are the usual hard-bounce classes. The last is ours: an address that exists but reaches the wrong people, such as a privacy or legal inbox that forwards to compliance rather than to anyone who buys software.

The verifier has two paths. The full one opens a connection to the recipient’s mail server and asks whether the mailbox exists, using the RCPT TO command from SMTP, then quits without sending a message. When that probe cannot run, a fallback does what it can without it: looks up the domain’s MX records, including a null MX, and checks the local part for role mailboxes and wrong-audience aliases.

What it actually did

The drop is a lookup in a cache file, not in the verifier’s output:

dead = [e for e in emails
        if cache.get(address(e["to"]), {}).get("status") in blocked]

The full path writes its verdicts to that cache. The fallback path computed each verdict, printed it, and returned without writing anything. Starting in mid-September the full probe could not run on our machine, so every run took the fallback.

For any address not already in the cache, which meant every new address from that point on, the lookup returned nothing. Nothing is not in the blocked set. So nothing was dropped. We measured it rather than inferring it: two test addresses, one that should pass and one that should be blocked as the wrong audience, each printed the correct verdict, and each had a null entry in the cache afterwards.

A gate that prints the right answer and acts on nothing looks exactly like a gate that works.

Why it never announced itself

The mailer prints the verifier’s output before it sends. So on every run, the correct verdict sat on screen directly above a send that went ahead anyway. Anyone reading the log saw a gate doing its job.

The blocked classes are also rare in our lists, because our sourcing already prefers published, non-role addresses on the company’s own domain. A gate that is rarely asked to block anything can be broken for a long time before the one case arrives where it matters.

That case arrived while we were building a batch. The one candidate we had in our target segment that day was a security company whose only address on its own domain was a privacy alias. We asked what the gate would do with it. The answer was: print WRONG_AUDIENCE, then send to it.

The fix: save the verdicts that block, and only those

The fallback now writes its blocking verdicts to the cache, tagged as fallback results. Invalid, no mail server, null MX, role alias and wrong audience are string checks on the local part plus DNS. None of them depends on the mail-server probe, so each is exactly as true from the fallback as it would be from the full path.

The one verdict the fallback deliberately does not save is the non-blocking one, which says only that the domain accepts mail. It is not a pass: it says nothing about whether the mailbox exists. Saved verdicts stay fresh for 30 days, so caching it would stop the real probe from running for a month once it works again. That would have traded this hole for a new one.

After the fix, the same two test addresses:

wrong-audience address  -> {"status": "WRONG_AUDIENCE", "mode": "dns-only"}
domain-accepts-mail     -> null    # not cached, by design

The security company was held, not sent. Its own site was searched for a buyer’s address first and none was published. That is the gate working as written, on the first batch where it could.

A write after the read is still a no-op

Fixing the writer was not enough to call it fixed. If the mailer had loaded the cache once at startup, before running the verifier, a verdict saved during the run would still be invisible to that run’s drop list. We checked: the mailer runs the verifier as a subprocess, then reads the cache file fresh in the same block, and exits rather than sending if that read fails. So a verdict saved for a first-seen address is acted on in the same run that produced it.

What to take from it

If a check has a fallback path, test the fallback on its own and look at what it leaves behind, not at what it prints. Ours printed perfectly. A degraded path is the one that runs when something else has already gone wrong, which makes it the one least likely to have been exercised and most likely to be running.

And watch for lookups that treat a missing answer as a pass. None not being in a blocked set is true, and it is not the same claim as “this address is fine”. It is the same shape as an empty result read as a successful request, and as a validator that checks what was asserted but not what was denied. Each time, the instrument returned a confident answer to a question it had not actually asked.

From our own engineering records, October 19, 2026. The fix shipped the same day, and it only adds to what the gate refuses. Addresses in the examples are described by type, not quoted.