SSH Key Management Guide: Keys, Certificates, Access

SSH keys are among the oldest and least governed non-human credentials in most organisations. They grant shell access to servers, network devices, Git repositories and cloud instances, often as root. Unlike passwords, SSH public keys in authorized_keys files never expire by default, are rarely tied to an owner, and survive the departure of the person or system that created them. Large estates can hold far more authorised keys than there are staff, many granting access nobody remembers approving. This guide covers how SSH authentication works, the risks of key sprawl, how to discover and clean up keys, and how SSH certificates and brokered access replace unmanaged keys.

Key takeaways

Every entry in an authorized_keys file is a standing access grant. Treat it like an account.
Discover first: find every authorised key, every private key and the trust relationships between them.
SSH certificates issued by a trusted CA with short lifetimes remove most key sprawl.
Broker privileged SSH through a bastion or PAM gateway with strong authentication and session recording.
Treat host keys and automation keys as NHIs with owners, scope and rotation.

How SSH authentication works

User keys: a client holds a private key; the server trusts the matching public key listed in a user's authorized_keys file.
Host keys: each server has its own key pair so clients can confirm they are talking to the right host; clients remember them in known_hosts.
SSH certificates: instead of listing individual public keys, servers trust a certificate authority. The CA signs short-lived certificates that state the user, allowed principals and expiry. Hosts can be certified the same way.
Agent forwarding: lets a remote host use your local keys for onward connections, which also lets an attacker on that host use them.

The risks

Key sprawl: keys copied to many servers and never removed. NIST IR 7966 describes how unmanaged keys create unknown trust relationships across an estate.
No expiry: plain public keys remain valid indefinitely.
No owner: keys left behind by leavers, contractors and retired automation.
Shared keys: one private key used by a team or copied into many scripts and CI systems.
Unprotected private keys: keys without a passphrase on laptops, in home directories, backups, container images and repositories. See the Leaked Credential Response Playbook.
Root and privileged access: keys authorised directly for root or service accounts with sudo rights.
Transitive trust: a compromised host whose keys unlock other hosts, enabling lateral movement.
Supply chain: SSH itself can be targeted. The XZ Utils backdoor aimed to subvert SSH authentication on Linux systems before it was caught.

Discovery and clean-up

Scan servers for authorized_keys files (including non-standard locations set in sshd_config) and record each key, the account it grants and its options.
Find private keys on servers, laptops, CI systems and code repositories, and map them to the public keys they match.
Map trust relationships: which private keys can reach which accounts on which hosts.
Assign owners and purposes; flag keys with no owner, no use in a defined period or root access.
Remove unowned and unused keys after a notice period, and disable direct root login.
Prevent regrowth: manage authorized_keys centrally (for example, via configuration management or AuthorizedKeysCommand) rather than letting users edit it freely.

Moving to SSH certificates

Configure servers to trust a user CA and stop accepting individual keys for people.
Issue certificates on demand after single sign-on and MFA, with lifetimes measured in hours.
Put the allowed principals (accounts or roles) in the certificate, so access is defined centrally.
Use host certificates so clients can verify hosts without trust-on-first-use prompts.
Protect the CA signing key in an HSM or managed KMS. See the Cryptographic Key Management Guide.
Plan revocation for the rare cases before a certificate expires, using revoked key lists.

Short-lived certificates are a form of just-in-time access. See the JIT Access Guide.

Brokered and privileged SSH

Route administrative SSH through a bastion or PAM gateway, so users never hold keys to production directly.
Record sessions for privileged access. See the Privileged Session Management Guide.
Consider cloud-native options that remove inbound SSH altogether, such as session manager services that authenticate with cloud identity.
Disable agent forwarding by default; use jump hosts (ProxyJump) instead.

SSH keys for automation and NHIs

Give each automation its own key or, better, a short-lived certificate issued to its workload identity.
Restrict automation keys in authorized_keys with options such as from= (source address) and command= (forced command).
For Git access from CI, prefer the platform's OIDC-based or short-lived tokens and deploy keys scoped to a single repository. See the CI/CD Pipeline Identity Security Guide.
Rotate automation keys and record owners like any other NHI. See the Service Account Security Guide.
Do not give AI coding agents access to personal SSH keys; issue them scoped, short-lived credentials. See the AI Coding Agents Security Guide.

Hardening basics

Use modern key types such as Ed25519; retire DSA and short RSA keys.
Disable password authentication where keys or certificates are in use.
Disable direct root login and require sudo with logging.
Keep OpenSSH patched and restrict who can reach SSH ports.
Log authentications, including which key or certificate was used.

Practitioner checklist

Discover all authorised keys and private keys, and map trust relationships.
Assign owners, remove unowned and unused keys, and stop direct root access.
Manage authorized_keys centrally.
Adopt SSH certificates with short lifetimes for people, then for automation.
Protect CA keys in hardware and plan revocation.
Broker and record privileged SSH access.
Restrict automation keys by source and command, and rotate them.
Scan code, images and backups for private keys.

SSH Key and SSH Certificate Management Guide