Skip to main content

Hardening, and a host-key surprise

Because the bastion is the one publicly-reachable SSH endpoint, it's the one that gets the most attention: keep it patched, restrict who can log in, prefer keys, watch its auth logs (they ship to the central log store like everything else).

A surprise worth writing down, though, came not from the bastion but from the VMs behind it. After rebuilding or even just rebooting some lab VMs, SSH would suddenly refuse to connect:

@@@  WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!  @@@

Alarming, but in this case benign: these VMs regenerate their SSH host keys on certain boots (a side effect of how they're provisioned from a cloud-init template). So the key genuinely changed, and SSH — correctly — refused until the stale entry was cleared:

ssh-keygen -R 10.100.100.x      # forget the old key, then reconnect

Lesson learned: a changed host key is SSH doing its job — it's supposed to shout, because a changed key can mean a man-in-the-middle. But know your environment: if you just reprovisioned the box, it's expected. The skill is telling "I rebuilt that VM" apart from "something is wrong," and never reflexively disabling host-key checking to make the warning go away.