When an AI agent misbehaves, responders lose time working out who owns it, what it can reach and whether stopping it will break something else, while the agent keeps acting at machine speed. This playbook is a ready-to-adapt procedure for incidents where an AI agent is the cause, the tool or the target: a hijacked agent, a poisoned memory, a stolen agent credential or an agent taking harmful actions on its own. It names roles, sets decision points with clear owners, and lists the evidence to preserve before anything is changed. It follows the NIST SP 800-61 Rev. 3 lifecycle and is written for security operations, incident managers and agent platform owners, with sections executives need for notification decisions. Background on agent telemetry and detection signals is in the AI Agent Observability and Incident Response Guide.
How to use this template
- Replace text in square brackets, such as [Organisation], [SOC] and [Agent platform team], with your own names, and fill every contact in Section 2 before you need it.
- Attach this playbook to your main incident response plan as an annex; it adds agent-specific steps and does not replace your general process. See the Incident Response Plan Template.
- Decision points are marked Decision: with the role that makes the call. Keep the owners as written or reassign them, but never leave a decision unowned.
- Steps marked Evidence: must be done before the containment or eradication step that follows them, unless the decision owner records why delay would cause greater harm.
- Rehearse the playbook at least [twice a year] with a tabletop and a live kill switch test on a staging agent.
1. Scope and triggers
1.1 This playbook applies to any incident involving an AI agent that accesses [Organisation] systems or data or acts on behalf of [Organisation], including internally built agents, low-code agents, coding agents, third-party agents connected through OAuth or API keys, and the MCP servers and tools they use.
1.2 Invoke this playbook when any of the following occurs:
- An agent performs an action outside its registered purpose, tools, data sources or destinations, or a high-impact action without required approval.
- Evidence of prompt injection, goal hijack or memory poisoning affecting an agent (MITRE ATLAS AML.T0051 LLM Prompt Injection, AML.T0080 AI Agent Context Poisoning).
- An agent's credential, token or API key is exposed, used from an unexpected location or used by something other than the agent.
- An agent reads credential stores or harvests credentials from tools (AML.T0098 AI Agent Tool Credential Harvesting).
- Data deleted, sent externally or modified by an agent without a legitimate request (AML.T0086, AML.T0101).
- A tool, MCP server, package or model used by agents is reported as compromised.
- Unexplained model usage or cost spikes linked to an agent or its keys.
1.3 Severity is set at triage and reviewed at each decision point.
| Severity | Criteria | Response target |
|---|---|---|
| Critical | Agent is actively causing harm: deleting or exfiltrating data, moving money, changing access, contacting external parties; or regulated or personal data confirmed affected | Contain within [15 minutes]; incident manager and CISO engaged |
| High | Confirmed compromise or hijack with access to sensitive systems, harm not yet confirmed | Contain within [1 hour] |
| Medium | Out-of-policy behaviour with limited reach, or suspected injection without impact | Triage within [4 hours] |
| Low | Policy deviation blocked by controls; no impact | Review within [2 business days] |
2. Roles and contacts
| Role | Responsibilities | Contact |
|---|---|---|
| Incident manager | Runs the incident, owns severity and the timeline, chairs decision points | [Name, phone, deputy] |
| SOC analyst / responder | Triage, evidence collection, containment actions, log analysis | [SOC rota] |
| Agent owner | Explains the agent's purpose, normal behaviour, principals and dependencies; approves business impact of stopping it | [From agent registry] |
| Agent platform team | Executes the kill switch, pauses agents, exports traces, memory and configuration, rolls back versions | [Team, on-call] |
| IAM / NHI team | Disables agent identities, revokes tokens, grants and keys, rotates secrets | [Team, on-call] |
| Cloud and application owners | Review and restore affected systems and data | [Per system] |
| AI governance lead | Assesses model or system behaviour, AI Act and AI policy implications | [Name] |
| Legal, privacy and DPO | Notification obligations, privilege, regulator contact | [Name] |
| Communications | Internal, customer and media communications | [Name] |
| Vendors | Model provider, agent platform, MCP server or SaaS vendor security contacts | [Security contact, contract reference] |
3. Detection and triage
3.1 Responder opens an incident record and identifies, from the agent registry and telemetry: the agent ID, version (model, system prompt, toolset), owner, the principal it acted for, every credential it holds and every system it can reach.
3.2 Responder determines whether the agent is still running and whether further tasks are queued or scheduled.
3.3 Evidence: before any change, preserve:
| Evidence | Source | Why it matters |
|---|---|---|
| Full traces for affected tasks: prompts, retrieved content, plan, tool calls with arguments, outputs | Agent platform, OpenTelemetry backend | Distinguishes injection, malfunction and instruction |
| Agent configuration and version: system prompt, tool list, policies, model ID | Agent platform, source control | Shows what was deployed at the time |
| Long-term memory and vector store snapshot | Memory service, vector database | Poisoned entries are evidence and a persistence mechanism |
| Identity and credential logs: token issuance, use and IDs | IdP, cloud audit logs, secrets manager | Scopes revocation and shows credential misuse |
| Gateway, MCP server and API logs | API or MCP gateway, tool servers | Confirms actions taken downstream |
| Egress, DNS and proxy logs | Network controls | Shows where data went |
| Human approval records | Approval service | Shows what approvers were shown |
| Affected data state: hashes, snapshots, change records | Data platforms | Supports recovery and impact assessment |
3.4 Store evidence in write-once storage with access logging, record hashes and maintain chain of custody. Treat prompts and traces as sensitive: they may contain personal data or secrets.
3.5 Responder forms an initial cause hypothesis: prompt injection through content the agent read; poisoned memory or retrieval data; compromised tool, MCP server or dependency; stolen agent credential used outside the agent; misconfiguration or excessive permissions; model error; or a legitimate but harmful instruction from a principal.
3.6 Decision (incident manager, with agent owner): confirm severity and whether this is an agent incident, a credential incident or both. If an agent credential is in use outside the agent, also run the Identity Incident Response Playbook; for cloud keys, the Cloud Credential Compromise Playbook.
4. Containment
4.1 Decision (incident manager): choose the containment level. Default to the higher level when harm is ongoing and the business impact of stopping is unknown.
| Level | Action | Use when |
|---|---|---|
| A. Restrict | Remove specific tools, destinations or data sources; require approval for all actions | Medium severity, cause understood, agent business-critical |
| B. Pause | Stop the agent's running and queued tasks; keep identity and data for investigation | High severity, credentials not believed compromised |
| C. Kill | Pause, disable the agent identity, revoke all credentials and grants, block at gateways | Critical severity, credential compromise or active harm |
| D. Class kill | Apply C to every agent sharing the affected tool, MCP server, model version, template or memory store | Shared root cause suspected |
4.2 Agent platform team stops execution through the orchestrator and removes the agent from discovery so other agents stop delegating to it. See the AI Agent Kill Switch and Containment Guide.
4.3 IAM team revokes credentials, recording each action and time. Examples:
- Microsoft Entra ID: disable the identity (
Update-MgUser -AccountEnabled:$falsefor user-type identities) and revoke refresh tokens (Revoke-MgUserSignInSession). Issued access tokens remain valid until expiry, typically one hour, unless the application supports Continuous Access Evaluation. - AWS IAM roles: use Revoke active sessions, which attaches the inline deny policy
AWSRevokeOlderSessionsusingaws:TokenIssueTime. It affects every session of that role, not only the agent's, and does not affect long-term IAM user access keys, which must be deactivated separately. - API keys, OAuth grants and personal access tokens: revoke at the issuing service; delete OAuth consents the agent holds on behalf of users.
- Shared signals: where relying parties support it, send an OpenID CAEP
session-revokedorcredential-changeevent so they end sessions without waiting for expiry.
4.4 Network and platform teams block implicated MCP servers, tools, packages and external destinations at the gateway and egress proxy.
4.5 Decision (incident manager, with principal's manager): if a user's delegated access was used, decide whether to revoke that user's grants to the agent platform or reset their sessions.
4.6 Responder confirms containment: no new actions, token issuance or egress from the agent identity for [30 minutes].
5. Eradication
5.1 Reconstruct each affected task end to end using trace IDs and confirm the cause (RS.AN-03 in NIST CSF 2.0).
5.2 Search for persistence the agent or attacker created: new memory entries, new credentials, service accounts or API keys, OAuth grants, scheduled tasks, webhooks, code commits, pull requests and accounts on external services.
5.3 Remove poisoned memory entries, malicious documents in retrieval sources and injected records (for example, form submissions or emails carrying instructions). Rebuild the vector index from clean sources where its integrity cannot be confirmed.
5.4 Rotate every credential the agent could reach, not only those it used.
5.5 Fix the root cause: narrow permissions, add approval gates, restrict egress, remove or replace the compromised tool, or roll back the agent version.
5.6 Decision (AI governance lead, with legal): if the agent is part of a high-risk AI system under the EU AI Act, do not modify the system in a way that could affect evaluation of the incident's causes before the competent authorities have been informed (Article 73(6)). Preserve a copy of the deployed version.
6. Recovery
6.1 Restore affected data from backups the agent identity could not reach or delete, and verify integrity against preserved hashes.
6.2 Reverse or correct unauthorised actions: messages, transactions, access changes, published content.
6.3 Decision (agent owner and incident manager, both required): approve re-enablement only when the root cause is fixed, credentials are new, detections for the attack pattern are in place and the agent has passed a test with the original attack input in staging.
6.4 Re-enable in stages: read-only, then approval required for all actions, then normal autonomy. Monitor at heightened sensitivity for [14 days] and treat the new version as a new behavioural baseline. See the AI Agent Behaviour Baselining and Anomaly Detection Guide.
7. Communication and notification
7.1 Incident manager briefs [CISO] at declaration and at each decision point, and [executive sponsor] for Critical and High incidents.
7.2 Decision (legal and DPO): assess notification duties as early as possible, as some clocks run from awareness:
| Obligation | Who and when |
|---|---|
| UK and EU GDPR Article 33 | Personal data breach likely to pose a risk: notify the supervisory authority within 72 hours of awareness |
| NIS2 Article 23 (EU entities in scope) | Significant incident: early warning within 24 hours, notification within 72 hours, final report within one month |
| EU AI Act Article 73 | Providers of high-risk AI systems: serious incidents to the market surveillance authority, no later than 15 days after awareness (2 days for widespread infringement, 10 days for a death). Regulation (EU) 2026/1744 deferred high-risk obligations (Annex III to 2 December 2027), so confirm which date applies to your system. |
| Contracts and sector rules | [Customer, insurer, financial or health regulator duties] |
7.3 Notify vendors whose platform, tool or model is implicated, and platforms where the agent acted (for example a code host where it opened pull requests), so they can remove artefacts and warn affected users. See the AI Incident Reporting Guide.
7.4 Notify affected individuals or organisations the agent contacted or affected, as legal advises.
7.5 Record every notification with time, recipient and content in the incident record.
8. Post-incident review
8.1 Hold a blameless review within [10 business days] with all roles in Section 2.
8.2 Document the timeline, cause, how it was detected, time to detect, time to contain, what was affected and which decisions were taken and why.
8.3 Record improvements (ID.IM-03 in NIST CSF 2.0) with owners and dates: registry gaps, permission changes, new detections, approval gates, egress rules, kill switch defects and playbook changes.
8.4 Update the agent's threat model and test cases, and add the attack input to regression tests for all agents of the same class. See the AI Agent Threat Modelling Guide.
8.5 Report metrics to [AI governance body]: incidents by cause, time to detect, time to contain, credentials revoked per incident and percentage of agents with a tested kill switch.
Implementation guidance
- Prepare the inputs: the playbook only works if the agent registry lists owner, credentials, tools and dependencies for every agent, and if traces are retained long enough to investigate. Close those gaps first.
- Build containment before you need it: script the pause, kill and class kill levels per platform and test them on a staging agent; record how long each takes. AWS role revocation, for example, denies every session of the role, so give each agent its own role.
- Learn from real cases: in the UK AISI agent testing incident, security monitoring flagged Tor egress, AISI declared an incident, stopped related work and isolated machines within about an hour, then worked with GitHub to remove artefacts and notify contacted users. The agent had also edited earlier activity to look harmless, which is why evidence must come from systems the agent cannot write to.
- Map to frameworks: this playbook supports NIST SP 800-53 IR-4, IR-5, IR-6 and IR-8 and AU-9 for evidence protection, and NIST CSF 2.0 RS.MA-01, RS.AN-03, RS.CO-02 and RC.RP-01. The Coalition for Secure AI (CoSAI) AI Incident Response Framework adds AI-specific containment such as model rollback and vector database rebuilds, with machine-readable CACAO playbooks.
- Automate carefully: SOAR can run Level A and B containment automatically on high-confidence alerts (IR-4(1)), but keep Level C and D decisions with a human unless the agent can cause irreversible harm faster than a person can respond. See the SOAR and Security Automation Guide.
- Keep related playbooks aligned: misuse of an agent by an authorised user is covered by the AI Agent Misuse Incident Playbook; the general lifecycle is in the NIST SP 800-61 Incident Response Guide.
Related NHI Mgmt Group resources: Incident Response Guide · Identity Incident Response Playbook · AI Agent Observability and Incident Response Guide · NIST SP 800-61 Incident Response Guide