If your jump host and bastion is compromised: containment, credential rotation and rebuild playbook

A compromised jump host or bastion hands the attacker the one path your administrators were told to trust, along with every credential and session that passed through it. Bastions sit between management devices and the systems being administered, which the NCSC calls an administration proxy: an SSH jump server, an RDP gateway or terminal server, or a reverse proxy in front of web consoles. Because they see many or all privileged sessions, they concentrate credentials, forwarded agents and live sessions in one place. The urgent questions are what the attacker could see and reuse, which downstream systems they reached, and how to restore administrative access without locking yourselves out. This playbook is for infrastructure, PAM, security operations and incident response teams. It is vendor-neutral and covers self-built SSH and RDP jump hosts as well as managed bastion services. Preventive design is in Securing the jump host and bastion itself.

Key takeaways

• Assume every credential that touched the bastion is exposed. Passwords typed in sessions, SSH keys stored on the host, forwarded agents and cached RDP credentials are all in scope.
• The blast radius is every system reachable from it. Scope the investigation by the bastion's network reach and the accounts used through it, not by the bastion alone.
• Keep administrators working through a clean path. Stand up a replacement bastion or break-glass route before you cut the old one, and alarm every break-glass use.
• Trust upstream logs over local ones. An attacker with root on the bastion can edit its logs and session recordings, so lean on firewall, identity provider and target-side logs.
• Rebuild, do not clean. Replace the host from a trusted image, re-issue host keys, and move to short-lived certificates or just-in-time access afterwards.

Scope: what a bastion compromise covers

If the bastion is part of a PAM product, run the If your privileged access management (PAM) vault is compromised playbook as well. If the incident began with stolen SSH keys elsewhere, use the SSH Key Compromise Response Playbook for the key estate.

How the compromise shows up

• Logins outside the normal pattern: admin accounts authenticating from new source addresses, at unusual times, or with an authentication method they never use.
• New keys or accounts: entries added to authorised keys files, new local users, or changes to sudoers and group membership on the bastion.
• Changed authentication configuration: edits to sshd configuration, PAM modules, trusted CA files or revocation lists, which can indicate T1556.
• Unusual forwarding: long-lived tunnels, dynamic forwarding, or connections from the bastion to destinations outside the documented admin target list.
• Logging and recording gaps: session recordings missing, audit daemons stopped or log volume dropping (T1685 Disable or Modify Tools).
• External signals: vendor advisories for your gateway, EDR alerts for credential-dumping tools, or a downstream system reporting admin actions no one requested. The Change Healthcare breach began with a remote access portal login that did not require MFA.

Triage: the first hour

1. Open a clean administrative path first: bring up a pre-built replacement bastion or the documented break-glass route, so responders are not forced through the compromised host. The NCSC warns that a proxy is a single point of failure and needs break-glass access with alarms on every use. See the Break-Glass Account Guide.
2. Capture volatile evidence: before shutdown, collect memory, the process list, active sessions, open network connections, running agent sockets and logged-in users.
3. Snapshot the disk: take a disk or VM snapshot and hash it, then export local logs, shell histories, session recordings and the bastion's configuration.
4. Isolate at the network layer: block inbound connections except from responders and block outbound connections from the bastion to all targets. Network isolation preserves the host for forensics better than shutting it down.
5. Pull upstream logs: export firewall and flow logs for the bastion's addresses, identity provider sign-ins for every account that used it, and authentication logs from reachable targets.

Scoping the blast radius

The question is not only what happened on the bastion but where it could reach. Build three lists and work from their overlap.

• Agent forwarding: any administrator who connected with agent forwarding enabled while the attacker had root could have had their agent used to authenticate onwards (T1563.001). Treat those users' keys as used, even though the private key never left their device.
• Typed secrets: passwords typed into sessions, including sudo and database passwords, may have been captured by a modified shell or keylogger.
• Cloud reach: if the bastion runs in a cloud account, check API activity for its instance role during the window. See the Cloud Incident Response Guide.

Containment by priority

1. Cut the attacker's sessions: terminate all sessions on the bastion and revoke identity provider sessions and refresh tokens for every account that used it in the window.
2. Revoke keys and certificates: add exposed user keys to the revoked keys list on targets (OpenSSH RevokedKeys accepts a plain key list or a KRL; replace the file atomically, because an unreadable file refuses all public key authentication). If you use an SSH CA, revoke certificates issued in the window.
3. Remove persistence on targets: sweep authorised keys files across reachable hosts for keys added during the window. See How to Find ssh authorized keys across a server fleet. The Cursor marketplace extension compromise planted an attacker SSH key for exactly this kind of persistence.
4. Tighten forwarding on the replacement: restrict PermitOpen to the documented host:port targets and disable agent forwarding. The sshd_config manual notes that disabling forwarding adds little unless shell access is also denied, so pair it with a forced command or no-shell design.
5. Reset privileged accounts: rotate passwords for local admin, root, domain admin and database accounts used through the bastion, starting with the most privileged.
6. Hunt on targets: investigate systems with actual connections from the bastion for new accounts, services, scheduled tasks and keys.

Credential rotation inventory

The GitHub RSA SSH host key exposure of 2023 shows the downstream cost of host key rotation: every client must accept a new key, so plan the known hosts update. See the SSH Protocol and Certificate Authority Guide.

Eradication and rebuild

1. Rebuild from a trusted image: do not clean the existing host. Build a new bastion from a hardened, patched image and apply reviewed configuration from source control. See the OpenSSH Server Hardening Guide.
2. Remove stored secrets: the new bastion should hold no private keys and no reusable passwords.
3. Move logs and recordings off-host: stream authentication logs, command audit and session recordings to a store that bastion administrators cannot alter (SP 800-53 AU-9).
4. Restrict inbound to trusted devices: the NCSC's guidance is to browse down from trusted management devices, never up. Require phishing-resistant MFA and device trust. See the Privileged Access Workstations Guide.
5. Restrict outbound: allow only the documented admin protocols to the documented targets.
6. Replace standing access: issue short-lived certificates or just-in-time grants per session. See the JIT Access Guide.

Recovery, testing and rollback

• Pilot ring: move one admin team to the new bastion first, confirm logging, recording and MFA work end to end, then migrate the rest.
• Negative tests: confirm a revoked key is refused on a sample of targets, forwarding to an undocumented destination fails, and a login from an untrusted device is blocked.
• Rollback: keep the break-glass route available until the new bastion has carried normal operations for an agreed period (NHI Mgmt Group recommendation: at least two weeks).
• Decommission: after evidence is secured, destroy the old host and remove its addresses, DNS names and firewall rules so nothing points to it.

Small and large organisations

Evidence and notification

• Preserve: memory and disk images, off-host logs and recordings, firewall and flow logs, identity provider sign-ins, configuration exports and target-side authentication logs, with hashes and chain of custody (CSF 2.0 RS.AN-07).
• Notification: whether regulators or customers must be told depends on what the attacker reached through the bastion. Assess against the Cyber Incident Reporting Requirements Map and coordinate with affected parties as NIST SP 800-61 Rev. 3 describes.

Detection and measurement

1. Admin sessions through the bastion from devices outside the trusted device list (target zero).
2. Private keys stored on bastion hosts (target zero).
3. Share of admin access using short-lived certificates or just-in-time grants rather than standing keys.
4. Session recordings and logs delivered off-host within the agreed delay (NHI Mgmt Group recommendation: 100% within five minutes).
5. Outbound connections from the bastion to undocumented destinations (target zero; each one investigated).
6. Break-glass uses without an alert and a post-use review (target zero).

Framework mapping

Common failure patterns

• Cutting the bastion with no alternative: administrators lose access during the incident and reopen unsafe direct routes.
• Trusting local logs and recordings: a root attacker can alter them; the absence of evidence on the host proves nothing.
• Rotating passwords but not keys: authorised keys added by the attacker on targets survive every password reset.
• Ignoring agent forwarding: users whose agents were forwarded through the host were usable as keys, even though their key files were never copied.
• Cleaning instead of rebuilding: a modified PAM module or sshd binary can survive a clean-up.
• Bastion trusted from untrusted devices: as the NCSC notes, a proxy alone does not stop a compromised management device from reaching admin interfaces through it.

Practitioner checklist

• Bring up a clean replacement bastion or break-glass route before isolating the old one.
• Capture memory, sessions and connections, then snapshot the disk and pull upstream logs.
• Build the reachable-systems, exposed-identities and actual-connections lists.
• Terminate sessions and revoke identity provider tokens for every account that used the bastion.
• Revoke exposed keys and certificates and sweep authorised keys files on targets.
• Rotate privileged passwords, cloud role credentials and integration secrets.
• Rebuild from a trusted image with new host keys, no stored secrets and off-host logging.
• Move to short-lived certificates or just-in-time access and track the metrics above.

Standards and references

• NCSC: Secure system administration
• NCSC: Protect your administration interfaces
• Microsoft Learn: Securing privileged access intermediaries
• OpenBSD: sshd_config manual page
• NIST: SP 800-61 Rev. 3 Incident Response Recommendations and Considerations for Cybersecurity Risk Management
• NIST: SP 800-53 Rev. 5 Security and Privacy Controls
• MITRE ATT&CK: T1563.001 SSH Hijacking
• MITRE ATT&CK: T1098.004 SSH Authorized Keys

Related NHI Mgmt Group resources: Securing the jump host and bastion itself · Privileged Access Management Guide · SSH Key Management Guide · If your VDI and DaaS is compromised · Jump host and bastion health check