How to Find ssh authorized keys across a server fleet

Every line in an authorized_keys file is a standing access grant, and on most server fleets nobody can say how many there are, who owns them or which ones still get used. Keys are added by hand during an outage, baked into images, pushed by old configuration management roles and, in intrusions, planted for persistence. This runbook shows how to build a complete, read-only inventory of authorised SSH keys across Linux and Unix servers: where sshd actually looks for keys, how to collect and fingerprint them at scale, how to tie each key to last use, and how to remove keys safely with a rollback. It is for infrastructure, platform and identity teams preparing a clean-up, an audit or a move to SSH certificates. For the wider key lifecycle, see the SSH Key Management Guide.

This is an independent guide from NHI Mgmt Group, based on the OpenSSH manual pages (sshd_config(5), sshd(8), ssh-keygen(1)) and NIST IR 7966 as of November 2026. Options differ between OpenSSH versions and vendor builds, so confirm against the manual on your own hosts.

Key takeaways

• Ask sshd where it looks, do not assume. AuthorizedKeysFile defaults to .ssh/authorized_keys .ssh/authorized_keys2, but it can be changed, set per user in a Match block, or replaced by an AuthorizedKeysCommand. Read the effective configuration with sshd -T.
• Collect every account, not just humans. Root, service accounts and accounts with nologin shells can still hold keys; the second file, authorized_keys2, is often forgotten.
• The fingerprint is the join key. Record a SHA256 fingerprint per key line, then match it to sshd login records, which log the same fingerprint, to separate used keys from dead ones.
• Options matter as much as the key. A key without from=, command= or restrict grants a full shell from anywhere; a cert-authority line trusts every certificate that CA signs.
• Remove in stages. Comment out or move a key with a recorded backup first, watch for failed logins, then delete. Restrict who can write key files so the sprawl does not return.

When you need this

• Audit or access review: auditors ask who can log in to production as root or as an application account. Authorised keys bypass directory groups and are invisible to most identity reviews. NIST IR 7966 notes that SSH user keys had received little security attention and treats the inventory as the first step of any key management programme.
• Leaver or vendor exit: a departed administrator or a terminated contractor may have left keys on dozens of hosts. Deleting their directory account does nothing to a key in /root/.ssh/authorized_keys.
• Suspected compromise: adding a key is ATT&CK technique T1098.004 SSH Authorized Keys, a simple persistence method. In the Cursor Marketplace Extension Compromise 2026 the tampered update tried to add an attacker key to authorized_keys with an innocuous-looking comment. Use the SSH Key Compromise Response Playbook alongside this runbook.
• Before moving to certificates: you cannot switch servers to TrustedUserCAKeys safely until you know which automation depends on plain keys. See the SSH Protocol and Certificate Authority Guide.

Before you start

Know every place sshd can take a trusted key from. The table lists the sources and how to read each one without changing anything.

Two behaviours affect what you find. With StrictModes yes (the default), sshd ignores a key file if the file, ~/.ssh or the home directory is writable by other users, so a key can exist on disk and not work; record it anyway, because a later permission fix would activate it. And AuthorizedKeysFile none means sshd reads no key files at all, which is common on hosts that use only certificates or a lookup command.

Prepare a read-only collection identity (an inventory agent, a configuration management fact, or a root-run collector with no write actions) and agree where the output lands. The inventory contains public keys only, but it maps who can reach which host, so store it as sensitive.

Step-by-step

Steps 1 to 5 are read-only. Step 6 changes access and is reversible. Step 7 is destructive.

1. Read the effective sshd configuration on each host. sshd -T checks the configuration file, prints the effective configuration and exits, with the same extra checks as sshd -t. Because Match blocks can change AuthorizedKeysFile per user, run it once per account class with a connection spec:

   sshd -T | grep -Ei '^(authorizedkeysfile|authorizedkeyscommand|authorizedkeyscommanduser|trustedusercakeys|authorizedprincipalsfile|revokedkeys|strictmodes|permitrootlogin)'
   sshd -T -C user=root,host=localhost,addr=127.0.0.1 | grep -i '^authorizedkeysfile'

   The -C keywords are addr, user, host, laddr, lport and rdomain. Record the output per host; a fleet where most hosts use the default and a few use a custom path is a common source of missed keys.
2. List every account and its home directory. getent passwd returns local and directory accounts the host can resolve. Keep root, system and service accounts in scope: a nologin shell stops an interactive shell but a key with a forced command, or one used for port forwarding, can still be useful to an attacker. See the Linux Service Accounts and Keytab Management Guide for the account side.
3. Collect each key line with its context. For every account, expand the configured paths and read each file that exists. Store one record per key line with these fields: host, account, UID, file path, file owner and mode, options, key type, key, comment, and a hash of the whole line. Lines that are empty or start with # are ignored by sshd; keep commented keys in a separate list because they are often re-enabled by mistake. If you run osquery, its authorized_keys table returns uid, algorithm, key, options, comment and key_file, and the project's example query is select * from users join authorized_keys using (uid). Check that it reads any custom AuthorizedKeysFile path you found in step 1 before relying on it alone.
4. Fingerprint every key. ssh-keygen -l -f <file> shows fingerprints, and -E selects the hash (sha256 is the default, md5 is available for old tooling). Run against an authorized_keys file, it prints one fingerprint per key line. Store the SHA256 value; it is how you will match keys across hosts and against logs.

   ssh-keygen -l -E sha256 -f /home/deploy/.ssh/authorized_keys

   If the host uses RevokedKeys or you keep a central KRL, test the collected files: ssh-keygen -Q -f revoked.krl authorized_keys returns zero only if no key in the file is revoked.
5. Join keys to last use. At the default LogLevel INFO, sshd records each successful public key login with the account, source address, key type and the key fingerprint, hashed with FingerprintHash (default sha256). Pull these records from your log pipeline (the journal or /var/log/auth.log or /var/log/secure, depending on the distribution) for a window that covers your slowest automation, then mark each inventoried fingerprint with its last login date and sources. Certificate logins log the certificate's key and CA, so keep them separate. The Linux Logging and auditd for Defenders Guide covers getting these logs off the host.
6. Classify and disable. Use the triage table below. For each key to remove, save the exact line and file metadata to your change record, then comment the line out or move it to a quarantine file outside the configured path. This is reversible in seconds. Watch for Failed publickey and service errors for an agreed period.
7. Delete and lock down. After the watch period, delete quarantined lines through configuration management, not by hand, so the change is recorded and repeatable. Then decide who may write key files (see Hardening afterwards).

Triage table

Reading the options field

The options before the key type decide what a key can do. Use these to score each line.

Choosing a collection method

A small team with a few dozen servers can do the whole job with a read-only script run from configuration management and a spreadsheet keyed on fingerprint. A large estate needs a scheduled collector feeding the NHI inventory, a log join that runs daily, and owners recorded per key. Network devices, appliances and cloud instance metadata also hold SSH keys and need their own collection route; see the NHI Inventory Reconciliation guide for merging sources.

Verify it worked

• Coverage: the number of hosts with a completed collection equals the number of hosts in your asset inventory that run sshd.
• Path completeness: every distinct AuthorizedKeysFile and AuthorizedKeysCommand value seen in step 1 has a collection method.
• Spot check: on a sample of hosts, compare the inventory against a manual read of each key file and the sshd -T output.
• Removal check: after step 7, re-run collection; removed fingerprints must not reappear, and no new Accepted publickey records should show them.

Detection and measurement

Inventory once is not enough: keys come back unless changes are watched. Monitor writes to key files and sshd configuration (an auditd watch or file integrity monitoring on /root/.ssh, home .ssh directories and /etc/ssh/sshd_config and its include directory), and alert on any new fingerprint that has no change ticket. The Linux File Integrity Monitoring Guide covers the tooling. Useful metrics:

• Authorised keys per host and per account: trend over time; spikes point to sprawl or tampering.
• Keys with an owner: percentage of fingerprints mapped to a named person or system owner, target 100%.
• Unused keys: fingerprints with no login in the window, trending to zero.
• Unrestricted automation keys: non-human keys without from= and command= or restrict.
• Root keys: count of keys on UID 0 accounts, target as close to zero as operations allow.
• Unapproved additions: new fingerprints per week without a change record, and mean time to remove them.

Rollback and common errors

• Automation breaks after removal: restore the saved line from the change record to the original file with the original owner and mode (user read and write only, as sshd(8) recommends), then check the job's log. Add the key to the inventory with an owner and restrictions before closing.
• Key restored but still refused: check StrictModes ownership and permissions on the file, ~/.ssh and the home directory, and check whether a RevokedKeys list now contains it. If the RevokedKeys file itself is unreadable, sshd refuses public key authentication for all users.
• Lockout of administrators: keep console or out-of-band access and a break-glass path tested before removing keys on root or admin accounts.
• Breaking sshd with a config edit: run sshd -t before reloading. If you add an AuthorizedKeysCommand, set AuthorizedKeysCommandUser too: sshd refuses to start without it.

Common failure patterns

• Scanning only ~/.ssh/authorized_keys: custom AuthorizedKeysFile paths, authorized_keys2 and lookup commands are skipped, so the riskiest keys are missed.
• Skipping service and system accounts: the inventory covers people and misses the automation keys with the widest reach.
• Treating the comment as identity: sshd does not use the comment field; anyone who can write the file can write any comment. Use fingerprints and change records.
• Deleting without a usage window: quarterly or annual jobs fail weeks later and the key is restored without restrictions.
• Clean-up without lock-down: users can still edit their own key files, so sprawl returns within months.
• Ignoring the private key side: removing a public key does not tell you where its private key lives. Search for identity keys on build servers and jump hosts too; see How to find unused GitHub deploy keys and SSH keys for the Git side.

Hardening afterwards

• Root-owned key locations: point AuthorizedKeysFile at a root-owned path per user (for example under /etc/ssh/authorized_keys/%u) managed only by configuration management, so users and compromised processes cannot add keys.
• Restrict automation keys: every non-human key gets restrict, from= and command=, and an expiry-time where the job allows.
• Move people to certificates: short-lived user certificates signed by a CA listed in TrustedUserCAKeys remove most per-host keys. See the OpenSSH Server Hardening Guide.
• Limit root login: set PermitRootLogin to no or forced-commands-only for reviewed exceptions. See Root, sudo and superuser accounts on Linux and Unix.

Framework mapping

Practitioner checklist

• Run sshd -T (with -C for each account class) on every host and record key-related settings.
• Collect key files for every account in getent passwd, including root, service accounts and authorized_keys2.
• Inventory keys served by any AuthorizedKeysCommand backend and any CA in TrustedUserCAKeys.
• Store one record per key line with host, account, path, options, type, comment and SHA256 fingerprint.
• Join fingerprints to sshd login records over a window that covers the slowest automation.
• Assign an owner to every key and flag shared, unrestricted and root keys first.
• Disable by quarantine with a saved copy, watch for failures, then delete through configuration management.
• Move key files to root-owned paths and monitor writes to key files and sshd configuration.
• Re-run collection on a schedule and alert on new fingerprints without a change record.

Standards and references

• OpenSSH: sshd_config(5) manual
• OpenSSH: sshd(8) manual, AUTHORIZED_KEYS file format and extended test mode
• OpenSSH: ssh-keygen(1) manual
• NIST: IR 7966, Security of Interactive and Automated Access Management Using Secure Shell (SSH)
• MITRE ATT&CK: T1098.004 SSH Authorized Keys
• NIST: SP 800-53 Rev. 5, Security and Privacy Controls
• osquery: authorized_keys table specification

Related NHI Mgmt Group resources: SSH Key Management Guide, SSH Key Compromise Response Playbook, How to Find credentials in cron jobs and scheduled tasks, NHI Dormancy Detection.