The carriage return that breaks the deploy, not the diff

Development on this repo happens on Windows. Deployment happens on Linux. Between them sits the oldest, quietest landmine in cross-platform work: the carriage return. A file written on Windows carries \r\n; the same file read by Linux tooling carries a stray \r that no diff viewer renders, no editor shows by default, and no code review catches -- because in every surface built for humans, the byte is invisible. The diff looks identical. That is the point of the title: the CR does not break the diff. It breaks the deploy.

The repo's defense landed in c7a7354 (PR #13): 35 lines of .gitattributes pinning LF endings on the files the server executes. The fix is right. What follows is a measurement of its reasoning -- because part of the reasoning survives contact with a Linux machine, and part of it, it turns out, does not.

The accident being prevented

Nothing about the repo's line endings was enforced before that commit. Every file under sys/ was stored with LF only because whoever committed it had core.autocrlf=true -- an accident of one machine's git config. A contributor with it unset would commit CRLF blobs, and the server would execute them.

The exposure is not hypothetical. Measured on the Windows dev box, with the rules now in place:

• 53 text files are CRLF on disk right now -- every unpinned text file, translated at checkout by core.autocrlf=true.
• 71 files are LF on disk -- the pinned set, plus files written by LF-emitting tools since their last checkout.

The rules make those two facts deliberate: the set of files that is LF on a Windows disk is now the set the server executes. Before the rules, that alignment was luck. (This note's own session contributed a specimen: two files sat with mixed endings on disk after edits, which git normalized on commit -- the system working, visible only to git ls-files --eol.)

What the CR actually does, per consumer

The commit asserted three failure modes. Each one is testable, so each one was tested -- on real Linux (WSL2 Ubuntu, systemd 255, nginx), with byte-exact files. The results sort into a spectrum, and the spectrum is the argument: the same bytes, harmless or fatal depending on who reads them.

Fatal: the shebang

The kernel parses #! itself, at execve, before bash exists to be lenient. It reads the interpreter path up to the LF -- so a CRLF script's interpreter is /bin/bash\r, and execve returns ENOENT for a file that differs from /bin/bash by one invisible byte. The classic message is the famous one:

/bin/bash^M: bad interpreter: No such file or directory

Modern bash words the same ENOENT as cannot execute: required file not found, which is arguably worse -- nothing in that sentence suggests line endings. Confirmed, exactly as the commit claimed.

Chaos: the bash body, as in an Actions run: block

GitHub Actions writes each run: block to a file and executes it with bash -eo pipefail. A fully CRLF script under those flags was run, byte-exact, line by line. What happens is not one failure but a cascade, and every step of it is worth naming:

• A bare word at end of line becomes a command name with a CR on it: false\r is not the builtin false, it is a lookup for a command that does not exist -- command not found, status 127.
• set +e\r prints set: +: invalid option -- and returns 0, and still disables errexit. The +e is applied before the parser chokes on the CR. The line that was supposed to turn errexit off does turn it off, while printing an error that makes you think it failed entirely.
• case "$status" in\r is a syntax error -- bash reports the token as $'in\r' -- and the script aborts there.
• The abort's exit status is whatever the last completed command left behind. The last completed command was the broken set, which returned 0. The script dies mid-work and exits 0.

Run the real vendor-pforex.yml block that way and the sequence is: the vendor check runs, status picks up its CR in the value (0\r -- the .gitattributes comment says "a variable whose name ends in a carriage return"; measured, the name never gets it, the value does), and then the case statement that would write drifted=true or drifted=false to $GITHUB_OUTPUT never executes at all. The step is green. The ::error:: line never fires. And the downstream step, gated on steps.check.outputs.drifted == 'true', compares an empty string and silently skips. The weekly automation does nothing, says nothing, and the log shows three errors on a green step.

The commit called this "fails in a way that looks like a logic bug rather than an encoding one." Measured, it is worse: it fails in a way that looks like success. This is the five-silent-failures class again -- the safety check whose own failure mode is silence.

Tolerated: systemd 255, and nginx

The commit said systemd units are "similarly unforgiving." Measured on the deploy platform's generation of systemd (255, Ubuntu LTS): a CRLF unit starts fine. The parser strips the CR as trailing whitespace -- ExecStart executes clean, and an EnvironmentFile with CRLF delivers FOO=bar to the process with no CR in it. nginx -t on a CRLF config: successful. Neither tool reproduces the claimed failure.

The pin for *.service and *.conf is still correct -- rules are cheap, older parsers exist, and defense in depth is free here -- but the claimed mechanism does not happen on the platform this repo deploys to. That distinction is the note.

Harmless, and drift-sensitive

notes.py reads markdown with open(path, encoding='utf-8') -- text mode, so Python's universal newlines normalize \r\n on the way in. The notes are deliberately not pinned: pinning them would add a rule whose only effect is churn. Same for the Python sources, the Svelte components, the stylesheets -- every consumer of those files normalizes or ignores the byte.

web/public/pforex.js is the special case, and the subtlest. It is vendored and must stay byte-identical to upstream. Its checker reads with universal newlines -- so a CRLF copy would compare equal and report "in sync" while the served bytes differ from upstream, byte for byte, on the one file in the repo with an exact-bytes requirement. The checker's tolerance is itself a small silence. The pin closes it: the ending stops depending on how each machine happens to check the file out.

Targeted, not blanket

The obvious alternative was * text=auto eol=lf -- one line, total coverage. The commit rejected it deliberately, and the measurement above is why the rejection is right: a blanket rule rewrites the entire working tree the moment it lands. On this machine that is 53 files of churn in one commit, and a permanent claim on files nobody executes -- the notes, the marketing doc, the Svelte sources -- none of which need protecting. The targeted rules instead make two sets coincide: LF on disk and the set the server executes. Nothing else is touched.

The renormalization check proved the rules bind only the future: git add --renormalize . staged nothing but .gitattributes itself -- every blob was already LF. Thirty-five lines, zero churn, and the accident they prevent is one git config away on any contributor's machine.

The fix was right. Half of its reasoning was folklore -- "systemd and nginx are similarly unforgiving" does not survive a systemd-analyze and an nginx -t on the platform that runs this site. That is worth writing down not to take the fix down a peg, but because it is the normal condition of preventive work: a preventive fix is justified by claims about failures that have not happened yet, and claims about failures are testable. This note is the test. The rules stay. The shebang alone would justify them; the green-step-with-missing-outputs alone would justify them. But the next time a defense is justified by three failure modes, the honest number is however many survive the attempt to reproduce them.

Three notes on failures whose only signature is silence:

• Five ways to fail without an error message -- five bugs, one repair commit, zero error messages. The taxonomy.
• The gitignore that ate a source file -- the silence that was manufactured: the tool that would warn you is the one told not to look.
• The carriage return that breaks the deploy, not the diff -- the invisible byte: the same bytes, harmless or fatal depending on who reads them. (this note)