SSH Hardening: Auditing and Restricting Authorized Keys for Critical Services

An SSH authorized_keys audit means listing every public key that can log in to your VPS, checking that each one is still needed, and restricting what the remaining keys are allowed to do. Most compromised servers are not broken into with a clever exploit — they are entered with an old key nobody remembered to delete. This guide shows you how to find every authorized_keys file, validate the keys inside, and lock each one down.

Before you start

You need:

• Root or sudo access on the Linux VPS.
• A second SSH session already open and logged in, so a mistake does not lock you out.
• The browser console in your VPS provider's control panel as a fallback — it works even when SSH does not.
• The fingerprints of the keys you personally use, so you can tell yours apart from unknown ones.

Find every authorized_keys file on the VPS

Start by asking sshd where it looks for keys. The default is .ssh/authorized_keys in the user's home directory, but some images and control panels change it.

sudo sshd -T | grep -i authorizedkeysfile

Now search the filesystem. -xdev keeps the search on the root filesystem so it does not wander into mounted volumes.

sudo find / -xdev -type f \( -name authorized_keys -o -name authorized_keys2 \) 2>/dev/null

Typical output:

/root/.ssh/authorized_keys
/home/deploy/.ssh/authorized_keys
/home/backup/.ssh/authorized_keys

If you keep data on a separate mount, repeat the search there without -xdev. Also check for keys stored outside home directories, which is common for service accounts:

sudo find /etc /opt /srv /var -type f -name authorized_keys 2>/dev/null

Write the list down. Every file you find is a door into the server.

Validate each key

For each file, print a fingerprint and comment for every key. The comment is the trailing text on the line and usually names the machine or person the key belongs to.

sudo ssh-keygen -lf /home/deploy/.ssh/authorized_keys

256 SHA256:9xQ... deploy@laptop (ED25519)
2048 SHA256:Kd7... old-ci-runner (RSA)

Count the key types too — a stray DSA or short RSA key is a red flag:

sudo awk '{print $1}' /home/deploy/.ssh/authorized_keys | sort | uniq -c

For each key, decide one of three things:

• Keep — you know the owner, it is still used, and it is an Ed25519 or RSA 3072+ key.
• Restrict — it is used by automation that only needs one command.
• Remove — nobody recognises it, it is unused, or it is a weak algorithm.

Restrict what a key can do

A key in authorized_keys can be limited with options placed before the key type. The restrict keyword turns off port forwarding, agent forwarding, X11 forwarding and PTY allocation in one go, and it is supported on current OpenSSH versions (Ubuntu 22.04/24.04, Debian 12, AlmaLinux and Rocky 8–9).

A backup key that should only run one script:

restrict,command="/usr/local/bin/backup.sh" ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... backup@example.com

A key that may only connect from your office range:

restrict,from="203.0.113.0/24" ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... deploy@example.com

A monitoring key that needs a shell but nothing else:

restrict,pty ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... monitor@example.com

Notes that matter in practice:

• command= replaces whatever the client asks to run. Interactive logins will drop straight into that command and exit.
• Options are comma separated with no spaces, and the whole option list goes on the same line as the key.
• A key can only have one line. Duplicate lines mean duplicate access — remove them.
• For stricter per-user policy, use a Match User block in sshd_config instead of editing keys by hand.

Fix ownership and permissions

OpenSSH silently ignores keys when permissions are wrong, which is why a correct key can still fail. Set them explicitly:

sudo chown -R deploy:deploy /home/deploy/.ssh
sudo chmod 700 /home/deploy/.ssh
sudo chmod 600 /home/deploy/.ssh/authorized_keys

The home directory itself must not be group- or world-writable:

sudo chmod g-w,o-w /home/deploy

Repeat for every account you found in the audit, including root and any service accounts.

Enforce and re-audit

Confirm the running configuration is what you expect, then reload:

sudo sshd -t
sudo sshd -T | grep -Ei 'pubkeyauthentication|permitrootlogin|authorizedkeysfile'
sudo systemctl reload sshd

On Debian and Ubuntu the unit is ssh, so use sudo systemctl reload ssh there. sshd -t validates the syntax before you reload — never skip it.

Make the audit repeatable. A short script that lists keys with fingerprints and owners, run monthly or from cron, catches drift early:

#!/usr/bin/env bash
for f in $(find /home /root -name authorized_keys 2>/dev/null); do
  echo "== $f"
  ssh-keygen -lf "$f"
done

Check it works

Open a new terminal and log in with the key you kept. Watch the negotiation with verbose output:

ssh -v deploy@203.0.113.10

You should see Offering public key, then Server accepts key, then a shell prompt. For a restricted key, confirm the restriction holds — try a port forward and it should be refused:

ssh -N -L 8080:localhost:80 backup@203.0.113.10

Then check the server log for the login:

sudo journalctl -u ssh -n 20

Troubleshooting

• Permission denied (publickey) after you changed permissions — the file or directory is owned by the wrong user, or the home directory is group-writable. Re-run the chown and chmod commands above.
• Key is present but never accepted — the line is wrapped across two lines, has a typo in the key blob, or lives in a file sshd does not read. Check AuthorizedKeysFile and keep each key on one line.
• A restricted key breaks an automated job — command= overrides the requested command. Point it at a wrapper script that validates the arguments, or use from= alone if the job needs a normal shell.
• sshd will not start after an edit — a syntax error in sshd_config. Run sudo sshd -t, fix the reported line, and reload again.
• Two fingerprints for the same person — an old key was never removed. Comment the stale line, confirm the new key still works, then delete it.

Next steps

• Audit and limit shell access for service accounts so automation keys cannot open an interactive shell.
• Enforce key rotation policies to stop keys living forever.
• Disable root login and password authentication once key access is clean.
• Set up SSH keys properly if you are still using passwords anywhere.