Two SSH keys, not one, and everything else a one-script VPS install actually needs

The goal was one script. Clone this repo, run install.sh as root, get a running site behind HTTPS. Nothing about that goal is exotic — it's the same shape as any small app's deploy story. Getting there took longer than it should have, not because any one piece was hard, but because of a handful of assumptions that don't announce themselves until the script actually has to run non-interactively, on a box already running other apps, against a private repo.

SSH keys authenticate a direction, not an identity

The script needs two SSH keypairs, and the confusing part isn't generating them — it's that they solve opposite problems and are easy to conflate into one "the deploy key" in your head.

The VPS needs to pull code from GitHub. That's one direction: the server proves its own identity to GitHub. The private half lives on the server; the public half is registered as a read-only deploy key on the repo.

ssh-keygen -t ed25519 -f /opt/app/.ssh/id_ed25519 -N "" -C "app-vps-pull"
# public half -> GitHub repo Settings -> Deploy keys (read-only)

CI needs to reach into the VPS to run the deploy script when something merges to main. That's the other direction: GitHub Actions proves its identity to the server. The private half is a GitHub Actions secret; the public half sits in the deploy user's authorized_keys.

ssh-keygen -t ed25519 -f ./ci_key -N "" -C "github-actions-deploy"
# public half  -> /opt/app/.ssh/authorized_keys on the VPS
# private half -> DEPLOY_SSH_KEY secret in the GitHub repo

You could technically use one keypair for both, but it means the same private key has to exist in two places — on the server and in GitHub's secret store — which is strictly worse than two separate keys. Leak either copy and both directions are compromised, instead of just one. Keeping them apart also means the pull key can stay read-only (it never needs to do anything but fetch), and the CI key's blast radius stops at this one server.

A correct key can still fail if the account's home directory has drifted

Both keys can be exactly right — matching fingerprints, correct permissions, authorized_keys sitting precisely where the deploy directory expects it — and sshd will still silently fall through to a password prompt. authorized_keys isn't looked up from wherever your scripts have been treating as home; it's looked up from whatever's actually recorded in /etc/passwd:

$ getent passwd appuser
appuser:x:1002:1002:,,,:/home/appuser:/bin/bash

useradd --home /opt/app only sets that field the moment the account is created. An account that already existed — from before this script existed, or an earlier partial attempt — keeps whatever home directory it originally got, and a later "user already exists, skipping" check has no reason to look twice. Every other step in the install script can use /opt/app consistently and never notice, because chown, systemctl, and git -C all take an explicit path — none of them go through the account's actual home field. SSH is the one thing that does, by default, and it fails quietly: no error about a mismatched path, just an authentication method that doesn't work for reasons that don't show up anywhere obvious.

CURRENT_HOME=$(getent passwd appuser | cut -d: -f6)
if [ "$CURRENT_HOME" != "$DEPLOY_DIR" ]; then
    usermod -d "$DEPLOY_DIR" appuser
fi

usermod -d refuses to run against an account with processes currently attached to it — stop the service first on a live box, or it fails with "user is currently used by process."

Certbot can't provision a certificate for a config that already needs one

The nginx vhost has to reference a certificate before it exists, except it can't, because certbot --nginx is what creates the certificate, and it works by finding and modifying an already-valid nginx config. Ship the HTTPS block with ssl_certificate pointing at a path Let's Encrypt hasn't populated yet, and nginx -t fails before certbot ever runs — the site can't even start serving HTTP, let alone get to the point of requesting a cert over it.

The fix is to not ship that block at all:

server {
    listen 80;
    server_name example.com www.example.com;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        proxy_pass http://127.0.0.1:8080;
    }
}

# certbot --nginx adds the `listen 443 ssl` block itself, once it has
# a cert to point at — including the HTTP -> HTTPS redirect.

Run certbot --nginx -d example.com -d www.example.com, and it finds that plain HTTP server block by server_name, gets a certificate for it, and rewrites the file with the HTTPS block filled in — cert paths, redirect, all of it. The config only ever needs to be valid for the step it's actually running.

A hardcoded port is a bet that this is the only app on the box

--port=8080 is a fine default until the VPS is running more than one service. On a shared box, whichever process binds the port first wins it; the second one crash-loops, restarting every few seconds against a port that's never going to free up.

The failure doesn't look like a port conflict from the outside. nginx is still faithfully proxying every request for the new site to 127.0.0.1:8080 — it's just that something else is listening there now, so the new domain quietly serves whichever app actually won the port. Confirming it just takes checking who's actually bound:

ss -tlnp | grep 8080

There's no clever fix here, just a boring one: pick a port that isn't already claimed on that specific box, and keep the systemd unit and the nginx proxy_pass in agreement about what it is.

"Skip if it's already there" isn't the same as "up to date"

An idempotent install script should be safe to run twice. The first pass at that idea here checked whether the deploy directory already had a git checkout, and skipped re-copying the code if so — which sounds right until you fix a bug, re-run the script, and the box doesn't change at all, because the check that guards the copy step doesn't distinguish "already set up" from "already set up, with a since-fixed bug baked in."

The better rule: sync from a fresh source checkout whenever one's available, and only fall back to what's already on the box if there's nothing fresher to copy from.

if [ -d "$SOURCE_DIR/.git" ]; then
    cp -r "$SOURCE_DIR/." "$DEPLOY_DIR/"
elif [ -d "$DEPLOY_DIR/.git" ]; then
    echo "no fresh source, using what's already deployed"
else
    echo "nothing to install from" && exit 1
fi

cp without --delete semantics only overwrites files that exist in the source — it never touches a destination file the source doesn't have an opinion about. That's exactly the property you want: .env, the virtualenv, node_modules all live only in $DEPLOY_DIR, so a resync never touches them, while everything actually tracked in git gets refreshed every time.

Anything a CI runner calls has to survive without a human at the keyboard

Every piece above still has to actually run unattended once a push to main is what triggers it, and each of the tools involved defaults to assuming a person is watching:

  • certbot will interactively ask about email, terms of service, and the HTTPS redirect unless told not to: --non-interactive --agree-tos --redirect --email you@example.com.
  • The first SSH connection to a new host prompts to confirm its key fingerprint. Pre-trust it instead of hoping someone's there to type yes: ssh-keyscan -t ed25519 github.com >> ~/.ssh/known_hosts.
  • A deploy script that restarts its own systemd unit needs to do it with sudo, and sudo over a non-interactive SSH session just hangs waiting on a password that will never come. One narrowly scoped sudoers rule fixes it without granting anything else:
appuser ALL=(ALL) NOPASSWD: /bin/systemctl restart appuser

None of these are hard individually. They just don't show up as bugs until the exact moment something tries to run the script without a human present to bail it out — which, for a script whose entire point is to eventually run from CI, is the only version of "working" that actually counts.

A deploy script that resets its own repo can un-executable itself

chmod +x on a freshly cloned script feels like a one-time fix. It isn't, if the deploy process's own update step is git fetch && git reset --hard origin/main — which is exactly the update step you want, since it guarantees the working tree matches origin exactly, no local drift, no merge conflicts to resolve unattended. The catch: git tracks a file's mode bit — executable or not — as part of the tree, same as its contents, and reset --hard restores both. A script committed as 100644 gets reset back to non-executable on every run that touches it, including, on the very next invocation, itself.

The run right after a manual chmod +x looks completely fine. The one after that fails with a plain Permission denied, and it reads like a sudo or ownership problem, because the script that would normally print something more useful is the exact thing that can no longer execute at all.

The fix belongs in git, not in the script:

git update-index --chmod=+x sys/scripts/deploy.sh
git commit -m "track deploy.sh as executable"

Once the mode bit is part of what's actually committed, reset --hard preserves it instead of stripping it — no runtime chmod step needed, and nothing left for the next reset to quietly undo.