CI tests three Pythons; production runs a fourth

Take a census of the Pythons in one small web project, the day this got fixed:

Every push ran the test suite on three interpreters. The one carrying production traffic was tested by none of them -- and neither was the one the developer actually edits in. Five Pythons, three green checkmarks, and the only row that matters got there by coincidence.

The coincidence is the problem

To be precise about the failure: it is not that CI tested the wrong version. Production's 3.12 sat inside the matrix, so the right version was covered. The problem is that nothing connected those two facts. The deploy box runs whatever python3 the distro ships -- chosen by apt on the box, never asked of the repo. The matrix is a list of guesses, one of which happens to be true. The day the box takes an LTS upgrade, the guesses stay exactly as green as they were and the truth moves underneath them, because CI cannot see the box at all. There is no link to break; that is why it breaks silently.

The matrix is the honest default -- for libraries

The matrix did not get into the workflow by carelessness. It is the community default, copied from project to project, and for the projects it was designed for it is correct. A library cannot know where its consumers run: the matrix is its deployment model, and testing 3.11 through 3.13 is how it keeps a promise it made to strangers.

An app is the opposite case and gets the same default anyway. An app knows exactly where it runs: one box, one venv, one interpreter. Its CI exists to predict that machine. Testing two machines that will never exist is coverage theater -- and it is not even free theater. It triples the CI minutes, it trains you to chase deprecation warnings for interpreters you will never deploy, and worst of all it feels like thoroughness while leaving the actual deployment untested. The checkmark distribution in the table above is what that feels like in practice.

The drift mode nobody can see

The deployment story makes the blind spot worse. install.sh builds the venv from the distro python3 exactly once. Every later deploy reuses that venv -- deploy.sh only runs pip when requirements.txt changed, because a redundant pip run costs seconds and a skipped one is worse.

Now let the distro upgrade its python3. Nothing rebuilds: same venv, same service, old interpreter, and no error anywhere to say so. CI's matrix does not notice -- it was never watching the box. The deploys do not notice -- they only diff file lists. The site keeps serving from an interpreter that is quietly no longer the one anyone tested, or meant to test. Every layer of this system is behaving exactly as written, and the answer to "which Python is production?" drifts away from every answer the repo gives.

One file, three consumers

The fix is to stop asking anyone to remember the version and write it down once, at the repo root:

.python-version   ->   "3.12"

Three consumers read it:

• CI installs it: setup-python with python-version-file: .python-version. The matrix is gone; the workflow has no version of its own to drift from.
• deploy.sh enforces it: after pulling, before restarting, it compares the venv interpreter against the file. A mismatch fails the deploy -- loudly, with the rebuild commands printed -- and leaves the current service running. That is the safe failure: the site keeps serving on the old interpreter while a human decides, instead of restarting into a half-migrated venv.
• DEPLOYMENT.md documents it, along with the drift mode above, so the next person who upgrades the box meets the contract before it bites.

Change the version deliberately and it is one commit: the file, plus whatever needs it beside it. CI follows, deploy follows, and the table at the top of this note collapses to two rows that agree.

What a pin cannot do

Honesty requires the last paragraph. A file in the repo is a claim, not a guarantee: the box still picks its own python3, and the deploy check fires after the drift exists rather than preventing it. The strong version of this fix -- provisioning the interpreter from the repo, the way some stacks vendor their runtimes -- would make drift impossible instead of visible, and is more machinery than one box on one distro has earned. The failure that used to be invisible is now loud, cheap, and carries its own repair instructions. For a web app, that is the whole difference between a version matrix and a version contract.

Keep the matrix for libraries. For an app: test the Python you ship, and make the repo -- not apt -- the place the version is decided.