Assume the host is compromised, isolate it, and preserve evidence before cleanup. Remove the package from dependency definitions, rebuild the environment from reviewed package names, and rotate SSH keys, cloud credentials, API keys, package-manager tokens, and Git credentials from a clean machine. Also review persistence entries and any suspicious startup artifacts added after install.
Why isolation and evidence preservation come first
The first move is to treat the host as actively compromised, not merely “suspicious.” That changes the response posture immediately: contain the machine, preserve volatile and non-volatile evidence, and avoid cleanup steps that destroy artefacts needed to understand what the package did, what it touched, and whether the compromise extended beyond the original install.
For Python package incidents, the initial compromise often sits inside a broader supply-chain event, so the immediate goal is to stop further execution and retain enough state to answer three questions: what was installed, what ran, and what access it gained. If the package landed credentials, persistence, or tampered with startup paths, the response now has to cover both the host and any downstream accounts or systems exposed by that host.
- Isolate the endpoint or server from normal network paths, but preserve the current state long enough to capture logs, process details, and package metadata.
- Record the package name, version, install time, source index, and the user or automation path that performed the install.
- Check for persistence artefacts before remediation erases them.
That containment-first approach is consistent with incident handling practice, and it is especially important when the package may have been used as a delivery vehicle for stolen developer or cloud credentials. See the incident-response coordination guidance from FIRST and the broader open-source supply-chain context in OpenSSF.
How to rebuild without reintroducing the compromise
After containment, rebuild from trusted inputs rather than trying to “clean” the existing environment in place. For package-driven compromises, the safest pattern is to remove the dependency from the manifest or lockfile, recreate the environment from reviewed package names, and verify every transitive dependency before the system is returned to service. That prevents a compromised site package, dependency typo, or malicious update from surviving a partial cleanup.
Credential hygiene is part of the rebuild, not a later optional step. Rotate SSH keys, cloud credentials, API keys, package-manager tokens, and Git credentials from a clean machine because a malicious package can read environment variables, config files, shell history, and local credential stores. If the host had access to build systems or deployment pipelines, extend the review to those paths as well, since the impact may be wider than the initial workstation or container.
- Recreate the environment from a reviewed requirements file or dependency lock, then compare the resulting package set against what was actually installed.
- Rotate any secret that could have been exposed on the host, even if there is no current evidence of abuse.
- Review startup items, scheduled tasks, shell profiles, service definitions, and package hooks for persistence added after the install.
For package provenance, dependency review, and secret exposure risk, the PyPI Breach case study and the LiteLLM PyPI package breach are directly relevant. For lifecycle and rotation discipline, the NHI Lifecycle Management Guide and Top 10 NHI Issues provide useful practitioner context.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK and OWASP Non-Human Identity Top 10 address the attack and risk surface, while CIS Controls v8 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| CIS Controls v8 | CIS Control 6 — Access Control Management | Compromised package installs often expose credentials that must be revoked and reissued. |
| CIS Control 5 — Account Management | The response requires reviewing and rotating accounts and secrets used on the affected host. | |
| CIS Control 17 — Incident Response Management | Isolating the host and preserving evidence are core incident-response actions after suspected compromise. | |
| Recommendation — Revoke exposed accounts, tokens, and keys, then reissue access from a clean admin path. Inventory affected accounts and remove any unnecessary credentials from the rebuilt environment. Contain the host, preserve artefacts, and coordinate recovery through your incident-response process. | ||
| NIST CSF 2.0 | RS.MI — Mitigation | The first steps are containment, eradication, and controlled recovery after suspected compromise. |
| RC.RP — Recovery Planning | Rebuilding the environment from reviewed package definitions is a recovery activity. | |
| Recommendation — Isolate the host, remove the malicious package, and restore from trusted sources. Rebuild the environment from verified dependencies and document restoration steps. | ||
| MITRE ATT&CK | T1552 — Unsecured Credentials | Malicious packages commonly seek stored SSH keys, API keys, tokens, and Git credentials. |
| T1053 — Scheduled Task/Job | Persistence checks should include scheduled tasks, startup hooks, and other autorun mechanisms. | |
| Recommendation — Hunt for exposed secrets and rotate any credentials accessible to the compromised host. Inspect autoruns and scheduled execution mechanisms for persistence added after installation. | ||
| OWASP Non-Human Identity Top 10 | NHI-01 — Secrets and Credential Management | The response centers on rotating exposed machine-access credentials after suspected package compromise. |
| NHI-02 — Authorization and Least Privilege | The package may have accessed overprivileged keys or tokens and broadened blast radius. | |
| Recommendation — Rotate exposed non-human credentials and rebuild using approved secret sources. Reduce privileges on credentials used by the affected host before returning it to service. | ||
Practitioner Guidance
What to verify: Before trusting any “clean” rebuild, verify that the dependency graph, startup state, and credentials in use all come from reviewed sources, not from the compromised host. If you cannot prove that a secret was unavailable to the malicious package, rotate it.
What to prioritise: Containment and credential rotation outrank root-cause analysis. The fastest safe path is to freeze the blast radius first, then reconstruct the environment from known-good package definitions, then investigate persistence and lateral exposure.
Common mistake: Teams often remove the package and redeploy from the same workstation or CI context that was already exposed. That preserves the compromise path and can reissue the same stolen tokens or keys into the rebuilt environment.
Practitioner takeaway: Treat the event as a host compromise with credential exposure until proven otherwise, because the response objective is not just removal of the package, but restoration of a trustworthy execution and secret state.
Related resources from NHI Mgmt Group
- What should security teams do first after discovering a collaboration account has exposed internal messages and credentials?
- What should teams do in the first 24 to 72 hours after discovering a compromised AI agent runtime?
- What should security teams do in the first 24 to 72 hours after a malicious package advisory?
- What should teams do in the first 24 to 72 hours after discovering agent misuse?
