Engineering
Three Ways zsh Made Our Checks Pass Without Measuring Anything
The same trap fired four times in one day. Each time a check reported success after measuring nothing.
A script was supposed to fetch a page from each of our sites, compare the bodies, and prove they were not all serving the same cached stub. It printed a green tick. It had fetched nothing. Three differences between zsh, the shell this machine runs, and the bash the script was written for had each turned a measurement into a no-op that still reported success. The same failure surfaced four times in one day before we accepted that the fix was not a better shell script. It was to stop writing verification in a shell at all.
The failure these checks are supposed to catch
We serve a plain-text .md twin of every article for machine readers. A caching bug can collapse all of them onto one body, so each URL returns whatever page happened to warm the cache first. The check for it is simple to state: fetch many URLs, and confirm the bodies are not all identical. We have written before about why a check like this needs a control. This post is about something smaller and more embarrassing: the check never ran, and said it had.
One: zsh does not split an unquoted variable into words
The script collected a list of URLs into a variable and looped over it:
urls=$(generate_urls); for u in $urls; do fetch “$u”; done
In bash, an unquoted $urls is split on whitespace, so the loop runs once per line. In zsh that splitting does not happen by default (the behaviour is controlled by the SH_WORD_SPLIT option, which is off). $urls expands to a single word containing every line joined by newlines, and the loop body runs exactly once, with the whole blob as the value of u. The zsh expansion rules are explicit about this; it is not a bug. The one fetch, of a newline-mashed non-URL, failed. Every body was empty.
An empty body has a checksum, and empty bodies all have the same one: the digest of empty input. So the comparison that was meant to tell the pages apart was handed a column of identical empty digests, and had nothing to tell apart. However it was phrased, it passed. Three separate runs printed a confident verdict on zero successful fetches. None of the verdicts was about any page.
The fix for this one line is to read the list with while read -r, which does not depend on word splitting. But that only repairs the loop. It does not repair the habit of trusting a shell to iterate.
Two: a glob that matches nothing aborts the whole command
A different check looked up a value in each site’s config by building a path with a glob:
grep key sites/$s/site.config.*
When a site had no file matching that pattern, bash would leave the pattern as a literal andgrep would simply fail to find it. zsh instead treats a non-matching glob as an error, the NOMATCH option, and aborts the entire command line before grep runs. Inside a loop over six sites, the first site with no matching config did not just skip itself. It killed the command, so all six produced the same UNKNOWN result, and a loop meant to measure six sites measured none of them.
The repair is to grep a directory with grep -r, or to set NULL_GLOB so a non-matching pattern expands to nothing instead of erroring. Again, local. Again, the next glob in the next script would do the same thing.
Three: the pipe-status array has a different name
The third check ran a command through a pipe and then read the exit code of the first stage:
curl … | grep …; code=${PIPESTATUS[0]}
PIPESTATUS is bash. In zsh the array is spelled pipestatus, lowercase, and indexed from one, as the zsh parameters manual documents. In zsh, ${PIPESTATUS[0]} is an unset variable. It expands to the empty string, so the check compared the empty string to its success value and drew whatever conclusion an empty exit code implied, which in practice was not failure.
Guarding each input is not guarding the verdict
We also did the obvious thing and wrapped every fetch in a guard that checked it had succeeded. One harness guarded all twelve of its fetches and still printed a green tick after zero of them succeeded. The guards were real. But the verdict was computed at the end from variables that were simply empty when every fetch had failed, and nothing between the guards and the verdict required that any guard had ever passed. Guarding the inputs told us each fetch was bad. It did not stop the script from reaching a conclusion as if they had been good.
This is the same shape as a gate that printed every verdict and saved none of them and treating an empty result as a measured zero. In all of them the thing that was checked was not the thing that mattered.
What we changed
Two rules, and neither is a shell idiom.
First: any script that fetches something and compares it runs in Python, with python3 -I, not in the shell. Not because Python cannot be written wrong, but because the specific failure that keeps biting us — an empty result silently joining, expanding, or vanishing into a passing comparison — is loud in Python and silent in a shell. An empty list has length zero that you can assert on. An unset name raises instead of expanding to nothing. The word splitting, the glob abort, and the renamed array are three doors into one room, and leaving the shell closes all three at once.
Second, and this is the one that generalises past any shell: require a minimum number of successful measurements before the script is allowed to print any verdict, and make falling short a loud failure, never a pass. All three bugs produced zero successful measurements. A check that refuses to say PASS or FAIL until it holds, say, twenty real readings would have said CANNOT TELL every time, which was the honest state. The worked example in our tree is a sweep that counts its own successful fetches and aborts to CANNOT TELL below a floor, so a run that measured nothing can never be mistaken for a run that measured zero problems.
The distinction in that last sentence is the whole point. “We checked and found no problem” and “we could not check” are different states, and a green tick collapses them into one. A red one at least makes you look.
The rule
Do not write verification logic in a shell, and do not let any verifier emit a verdict without first proving it took enough real measurements to have one. A shell will happily run a loop zero times, abort halfway through, or read a variable that does not exist, and report success through all of it. The cost is not a crash you would notice. It is a column of identical empty digests that looks exactly like a clean result.
From our own engineering records, November 4, 2026, when the same trap fired four times in a single day across two of us. Shell and tool names are standard; no prospect or client is named.
Frequently asked
Why does an unquoted variable not split into words in zsh?
It is a deliberate design choice. In bash and other POSIX shells, an unquoted expansion like $urls is split on whitespace into separate words, which is why a for loop over it iterates once per line. zsh does not do this by default, so the same $urls expands to a single word containing every line. The loop body runs exactly once, with the whole blob as one value. You can opt back in with the SH_WORD_SPLIT option, but relying on a non-default option for correctness is its own trap.
What is the nomatch behaviour?
When a glob like sites/*/config.json matches no files, bash by default leaves the pattern as a literal string. zsh instead treats a non-matching glob as an error and aborts the entire command line before it runs. Inside a loop that greps a path built from a glob, one site with no matching file does not skip that site; it kills the command, and every site after it is never measured.
What is the difference between PIPESTATUS and pipestatus?
Both hold the exit codes of the commands in the last pipeline, one per stage. bash spells the array PIPESTATUS in uppercase. zsh spells it pipestatus in lowercase and indexes from 1, not 0. A script written for bash that reads ${PIPESTATUS[0]} in zsh reads an unset variable, which expands to nothing, so a check that depends on it concludes nothing or silently concludes success.
Why not just fix the shell script?
Because the failure mode is a check that passes while measuring nothing, and a shell makes that specific failure easy and quiet in several independent ways. We could have fixed each one, but the next quirk would have produced the same green tick. The durable fix is to do the fetching and comparing in a language where an empty result is an object you can count, not a string that silently joins with the next one. We moved that logic to Python run with -I.
What is the one rule that would have caught all three?
Require a minimum number of successful measurements before emitting any verdict, and make the shortfall a loud failure rather than a pass. All three bugs produced zero successful measurements. A check that refuses to say PASS or FAIL until it has, say, 20 real readings in hand would have said CANNOT TELL in every case, which is the true state.
How were the digests all equal if nothing was fetched?
Every fetch returned an empty body, so every checksum was the digest of empty input, which is the same value every time. The comparison that was meant to tell the pages apart was handed a column of identical empty digests and had nothing to tell apart, so it passed. The verdict was not about the pages. It was about a column of identical empty digests.