Security

SSH Hardening: Mitigating Man-in-the-Middle Attacks with Host Key Verification

SSH host key verification makes your client confirm the server's identity before you send credentials. This guide shows how to read fingerprints, verify them out of band, pin them in known_hosts and rotate host keys without locking yourself out.

IntermediateLinux server6 min read

SSH host key verification is what stops an attacker on the network path from impersonating your Linux VPS and collecting your password or session. Your client compares the key the server presents against the fingerprint it already trusts, and refuses to continue when they differ. This guide walks through reading host key fingerprints on the server, confirming them through a second channel, pinning them in known_hosts, and rotating keys when you have to.

Before you start

You need:

  • A Linux VPS running Ubuntu 22.04/24.04, Debian 12, AlmaLinux or Rocky 8–9, with root or sudo access.
  • An SSH client on your workstation. OpenSSH ships with Linux, macOS and Windows 10/11, so ssh-keygen and ssh-keyscan are already available.
  • Access to your VPS provider's browser console. This is your out-of-band channel — it does not travel over the same SSH connection you are verifying.
  • Key-based login already working, so a rejected password prompt never costs you access. See Set up SSH keys if you have not done that yet.

Note: Host key verification protects the server's identity. It is separate from your own user key, which proves your identity to the server. Both matter.

How host keys stop a MITM

Every SSH server holds one or more host key pairs in /etc/ssh/, typically Ed25519, RSA and ECDSA. When a client connects, the server signs part of the handshake with the private host key and sends the public key. The client looks that key up in ~/.ssh/known_hosts.

On the first connection there is nothing to compare against, so OpenSSH asks you to trust the key — this is trust on first use. That prompt is the weak point: if an attacker sits between you and the VPS and you type yes without checking, you have just pinned the attacker's key instead. Verification means comparing the fingerprint against a source the attacker cannot touch.

Step 1 — Read the host key fingerprints on the server

Log in and list the public host keys:

Bash
ls -l /etc/ssh/ssh_host_*_key.pub

Print the SHA256 fingerprint of each one:

Bash
ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub
ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub

Expected output looks like this:

256 SHA256:8Qn1vXk3pR7tYb2LmWc0dHs4ZaE9fJg6NhUu1oPqRsA root@vps (ED25519)

Confirm which keys the daemon actually offers:

Bash
sudo sshd -T | grep -i hostkey

You should see one hostkey line per configured key. Anything you do not recognise deserves a look before you continue.

Step 2 — Verify the fingerprint out of band

Open your VPS provider's control panel, start the browser console for your VPS and run the same ssh-keygen -lf command there. Compare the SHA256 string character by character with what your local client shows.

Warning: Never verify a fingerprint using the same SSH session you are trying to validate. If that session is already intercepted, the attacker controls both sides of the comparison.

If you manage DNS for the domain, publishing SSHFP records lets clients validate keys automatically. Details change between OpenSSH releases, so check the vendor documentation for the current record format and your registrar's interface.

Step 3 — Pin the host key in known_hosts

On your workstation, scan the key and inspect it before trusting it:

Bash
ssh-keyscan -t ed25519 203.0.113.10 > /tmp/vps-hostkey
ssh-keygen -lf /tmp/vps-hostkey

Compare that fingerprint with the one from Step 2. Only when they match, append it:

Bash
cat /tmp/vps-hostkey >> ~/.ssh/known_hosts

Warning: ssh-keyscan is unauthenticated. It fetches whatever key the network hands back, so it is exactly as trustworthy as the connection you are trying to secure. Never append its output without an out-of-band check.

Check what your client already stores, and remove stale entries:

Bash
ssh-keygen -F 203.0.113.10
ssh-keygen -R 203.0.113.10

Step 4 — Enforce strict host key checking

Edit ~/.ssh/config and create an entry for the VPS:

Host vps
    HostName 203.0.113.10
    User deploy
    StrictHostKeyChecking yes
    UserKnownHostsFile ~/.ssh/known_hosts
    HashKnownHosts yes

With StrictHostKeyChecking yes, the client refuses to connect when the key is unknown or has changed. That is what you want for anything you manage by hand. For automated jobs that connect to a freshly built server, accept-new pins the key on first contact and still fails on later changes — a reasonable middle ground. Set the same options system-wide in /etc/ssh/ssh_config if you prefer.

Step 5 — Rotate host keys safely

Rotate when a key is exposed, when you rebuild the OS, or on a schedule that matches your policy.

Warning: Regenerating a host key changes its fingerprint. Every client that already trusts the old key will show a loud warning until you redistribute the new one. Keep an existing SSH session open while you work so a mistake does not lock you out.

Bash
sudo ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N ""
sudo ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ""
sudo chmod 600 /etc/ssh/ssh_host_*_key
sudo chown root:root /etc/ssh/ssh_host_*_key
sudo systemctl restart ssh

On AlmaLinux and Rocky the unit is sshd:

Bash
sudo systemctl restart sshd

Then repeat Steps 1 to 3 to publish the new fingerprints to your team and your automation. If you use SELinux, restore the file contexts after replacing keys:

Bash
sudo restorecon -v /etc/ssh/ssh_host_*

Check it works

Run a verbose connection and read the host key lines:

Bash
ssh -v -o StrictHostKeyChecking=yes deploy@203.0.113.10

You should see the key type, the fingerprint and a line confirming the match against known_hosts, followed by a normal login. To prove the protection is real, deliberately corrupt the stored entry on a test machine:

Bash
ssh-keygen -R 203.0.113.10
printf '203.0.113.10 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBogusKeyForTestingOnly\n' >> ~/.ssh/known_hosts
ssh deploy@203.0.113.10

The client must stop with REMOTE HOST IDENTIFICATION HAS CHANGED and refuse to send credentials. Remove the fake line with ssh-keygen -R 203.0.113.10 afterwards.

Troubleshooting

  • REMOTE HOST IDENTIFICATION HAS CHANGED — either you rotated the key or someone is intercepting the connection. Verify the new fingerprint through your VPS provider's console first, then clear the old entry with ssh-keygen -R 203.0.113.10 and reconnect.
  • Host key verification failed — StrictHostKeyChecking yes is set but no entry exists. Add the verified key to known_hosts rather than weakening the setting.
  • No ED25519 host key is known — the server does not offer that algorithm. Scan the type it does offer, for example ssh-keyscan -t rsa 203.0.113.10, and check the HostKey lines in /etc/ssh/sshd_config.
  • You were prompted for a password after a warning — the warning was accepted blindly and the connection continued. Treat the credentials you typed as compromised and rotate them.
  • grep finds nothing in known_hosts — entries are hashed. Use ssh-keygen -F 203.0.113.10 instead of searching the file.
  • sshd will not start after key rotation — check ownership and permissions on /etc/ssh/ssh_host_*_key and run restorecon on SELinux systems.

Next steps

Was this guide helpful?

Still stuck?

Ask MdmHost AI for a quick answer, or open a ticket and our team will take a look. We are here 24/7.

Ask MdmHost AIOpen a ticket

Related guides

Security

Set up a UFW firewall on Ubuntu and Debian

Set safe defaults, allow SSH before switching UFW on, open only the ports you use, limit sensitive ports to your own IP, and tidy up old rules.

3 min read
Security

Make iptables rules survive a reboot

iptables rules live in memory and disappear on reboot. Save them with netfilter-persistent on Debian and Ubuntu or iptables-services on AlmaLinux and Rocky Linux.

5 min read