Join our Newsletter — 33% off our NHI Course
Home› Guides› Prompt Injection Incident Response Playbook
Template Agent Threats & Attacks

Prompt Injection Incident Response Playbook

PDF for newsletter subscribers · free to read online

← All guides
By NHI Mgmt Group, edited by Lalit Choda Updated 13 November 2026 12 min read
On this page

When an AI agent or assistant has acted on instructions hidden in content it read, the incident is not over when you stop the agent: the poisoned content is still sitting in a mailbox, a CRM field, a web page or a vector store, waiting for the next agent to read it. This playbook covers the specific case of a confirmed or suspected prompt injection, direct or indirect, that caused an agent, copilot or LLM application to leak data, call tools, change records or send messages it should not have. It is a fill-in template: replace the bracketed placeholders and attach it to your incident response plan as an annex. General agent incidents (runaway cost, stolen agent credentials, malicious MCP servers) are covered by the AI Agent Incident Response Playbook and the AI Agent Misuse Incident Playbook; this page adds the steps that are particular to injected instructions.

How to use this template

  • Customise: replace every [Placeholder] with your teams, tools and contacts, and delete rows that do not apply to your estate.
  • Decisions: steps marked Decision: need a named owner, a recorded outcome and a timestamp in the incident log.
  • Assume residual risk: the UK NCSC states that prompt injection may never be totally mitigated, because current LLMs do not enforce a boundary between instructions and data. Plan for recurrence, not for a one-off fix.
  • Rehearse: run a tabletop at least [twice a year] using a planted, harmless injection in a test mailbox or document library, and time each phase.
  • Keep it short in the moment: the numbered sections are the run sheet; the Implementation guidance section is preparation work to do before you need it.

1. Scope and triggers

This playbook applies to [Organisation]'s LLM applications, copilots, RAG assistants and autonomous agents, whether built in house, embedded in SaaS (for example CRM or office suite assistants) or used through a provider API. Prompt injection is MITRE ATLAS AML.T0051 LLM Prompt Injection, with sub-techniques AML.T0051.000 Direct and AML.T0051.001 Indirect, and OWASP LLM01:2025 in the Top 10 for LLM Applications.

TriggerTypical signalInitial severity
Agent output contains a URL or image link carrying data in its query stringEgress or proxy log shows requests to an unfamiliar domain right after an agent response rendersHigh (possible exfiltration, ATLAS AML.T0085 Data from AI Services)
Agent called a tool the task did not needTool-call trace shows email send, file share, record update or code execution outside the user's request (AML.T0053 AI Agent Tool Invocation)High if the tool writes or sends; Medium if read only
Data sent out through a tool callOutbound message, webhook or API call containing records the user did not ask for (AML.T0086 Exfiltration via AI Agent Tool Invocation)Critical where personal or regulated data is involved
Instruction-like text found in content the agent readsContent scanner, classifier (for example a prompt-shield service) or user report flags hidden text in email, tickets, form fields, PDFs or web pagesMedium until you confirm whether an agent processed it
Poisoned memory or retrieved contextAgent repeats an instruction across sessions, or a vector store chunk contains directives (AML.T0080 AI Agent Context Poisoning)High: affects every user of that memory or index
Vendor or researcher disclosureAdvisory for an assistant you use, as with the EchoLeak Microsoft 365 Copilot vulnerabilitySet by exposure: did the vulnerable feature process external content in your tenant?

Severity guide: Critical = confirmed exfiltration of regulated data or an irreversible action (payment, deletion, external message). High = tool misuse or data exposure inside [Organisation]. Medium = injection present and processed, no harmful action found yet. Low = injection found in content no agent has read. Response targets: Critical [15 minutes to containment decision], High [1 hour], Medium [same business day].

2. Roles and contacts

RoleResponsibility in a prompt injection incidentContact
Incident managerRuns the incident, owns the log and every Decision step[Name, phone]
SOC analystPulls prompt, response and tool-call traces; egress and proxy logs; builds the timeline[SOC rota]
Agent or application ownerKnows the system prompt, tools, data sources and memory; executes containment on the agent[Owner per agent, from the agent registry]
Content source ownerOwns the system the injection came from (mail, CRM, document library, website, ticketing) and quarantines the carrier[Per source]
IAM or NHI teamRevokes and reissues the agent's tokens, OAuth grants and API keys; reviews what the agent's identity can reach[IAM on call]
AI platform or ML engineerInspects retrieval, memory and vector stores; deploys guardrail and filter changes[Platform team]
Data protection officer and legalDecides on GDPR and contractual notification; advises on regulator and customer wording[DPO, counsel]
SaaS or model providerProvider-side logs, fixes and confirmation for embedded assistants[Support route, contract reference]

3. Detection and triage

  1. Preserve before you change anything. Export the full conversation or task trace: user prompt, system prompt version, retrieved chunks with their source IDs, tool calls with arguments and responses, model output as rendered, and timestamps. Snapshot the agent's memory store and configuration. Hash the exports and record where they are stored.
  2. Find the carrier. Identify the exact content that held the injected instruction: the email message ID, CRM record and field, document path and version, URL and fetch time, ticket number or tool response. Copy the payload text into the evidence store as inert text (for example in a .txt file) and never paste it into another LLM tool for analysis.
  3. Classify the path. Direct (a user typed it), indirect (the agent read it from external content), or stored (it came from memory or a vector index that earlier content poisoned). The path decides who else is exposed.
  4. Reconstruct actions. List every tool call and output made after the carrier entered context. For each: what it did, which identity and scopes it used, whether it can be reversed.
  5. Check for outbound channels. Search agent outputs for links, Markdown images (including reference-style syntax), HTML, and calls to webhooks or messaging tools. Correlate with proxy, DNS and CDN logs to see whether a client or server actually fetched them.
  6. Measure fan-out. Query for every other agent, user session and scheduled job that retrieved the same carrier or index chunk in the exposure window. Injected content in a shared mailbox, CRM object or knowledge base usually reaches more than one session.
  7. Decision: confirm severity from section 1 and whether this is a vulnerability in the product (vendor fix needed), a design gap in [Organisation]'s agent (excessive tools or access), or both.

4. Containment

ActionHowWhen
Quarantine the carrierMove or tag the email, lock or redact the CRM field, unpublish or exclude the document, block the URL from retrievalAlways, first
Remove poisoned content from indexes and memoryDelete affected chunks from the vector store by source ID; clear or roll back the agent's long-term memory to a pre-incident snapshotStored or indirect paths
Close the exfiltration channelBlock the destination domain at proxy and DNS; disable external image and link rendering in agent output; restrict the content security policy and URL allowlist to domains you controlAny outbound signal
Restrict the agentRemove write and send tools, switch to read only, or require human approval for every actionHigh
Pause or kill the agentUse the kill switch described in the AI Agent Kill Switch and Containment GuideCritical, or when fan-out is unknown
Revoke the agent's credentialsRevoke OAuth grants, refresh tokens and API keys the agent used; reissue only after eradicationAny confirmed tool misuse

Quarantining the carrier comes before stopping the agent, because stopping one agent does not stop a second agent, a scheduled job or a colleague's copilot from reading the same content. In the ForcedLeak Agentforce case the payload sat in a public Web-to-Lead Description field until an employee asked the agent to process the lead, and data left through an image URL on an allowlisted domain that had expired.

5. Eradication

  • Hunt for siblings: search every source the agent reads for the payload, its distinctive phrases and variants (other languages, encodings, white-on-white text, document metadata, alt text). Payloads are often split across several items so that each looks harmless.
  • Write a detection from the payload: add the indicators to content scanning and to the classifier tuning set, accepting that rephrased payloads will evade phrase lists.
  • Fix the design gap: remove tools the task does not need, scope the agent identity to the minimum data, and require approval for irreversible or external actions. See the Human-in-the-Loop for AI Agents Guide.
  • Separate untrusted input: mark external content as data in the prompt (delimiting, datamarking or encoding, the spotlighting approaches Microsoft describes) and keep agents that read untrusted content away from privileged tools.
  • Rotate what was reachable: any secret, token or document link that appeared in the agent's context during the exposure window should be treated as disclosed.
  • Apply vendor fixes: for embedded assistants, obtain written confirmation of the provider-side fix and any tenant settings you must change.

6. Recovery

  1. Reverse unauthorised actions: recall or correct messages sent, restore changed records from audit history or backup, cancel scheduled tasks the agent created.
  2. Rebuild affected indexes from clean sources rather than patching them in place, if you cannot prove which chunks were poisoned.
  3. Replay the original payload and a set of variants against the fixed agent in a staging environment. Re-enable only when none trigger a tool call or outbound request.
  4. Decision: approve return to service, with reduced scope where appropriate, by the agent owner and the incident manager.
  5. Run heightened monitoring for [14 days]: alert on any tool call outside the agent's baseline and on any output containing external URLs.

7. Communication and notification

  • Internal: brief affected users on what the agent did in their name and what they should check, such as sent items or shared files.
  • Personal data: if personal data left [Organisation], the DPO assesses GDPR Article 33 notification to the supervisory authority within 72 hours of becoming aware, and Article 34 notification to individuals where the risk is high. See the Data Breach Notification Checklist.
  • Sector and AI rules: check sector incident reporting (for example DORA or NIS2) and whether the system falls under EU AI Act incident obligations. See the EU AI Act Provider vs Deployer Obligations Guide.
  • Providers: report the injection path to the SaaS or model provider through its security disclosure route, with the payload as inert text.
  • Accuracy: say "an assistant followed instructions hidden in [source]", not "the AI was hacked", and do not state that no data left until egress logs confirm it.

8. Post-incident review

  • Hold a blameless review within [10 working days] with the agent owner, content source owner and SOC.
  • Answer: which untrusted source could reach the agent, why the agent had the tool that was misused, why the output channel was open, and how long the carrier sat before detection.
  • Update the agent's threat model with the path observed. See the AI Agent Threat Model Template.
  • Add the payload and variants to regression tests that run on every prompt, tool or model change.
  • Record time to detect, time to quarantine the carrier and number of sessions exposed, and report them to the AI governance forum.

Implementation guidance

Prepare before an incident

  • Trace logging: retain prompts, retrieved chunk IDs, tool calls and rendered outputs for at least [90 days]. The NCSC recommends logging inputs, outputs, tool use and API calls, including failed tool calls, which can signal an attack in progress.
  • Source lineage: every chunk in a vector store must carry its source ID and ingestion time, or you cannot delete poisoned content selectively. See the RAG Security Guide.
  • Agent registry: owner, tools, data sources and identity for every agent, so triage does not stall.
  • Output controls: disable rendering of external images and links in agent output by default and allow only domains you own and monitor.
  • Containment switches tested: read-only mode, approval mode and full stop for each agent, rehearsed.

Framework mapping

FrameworkReferencePlaybook section
NIST SP 800-61 Rev. 3 (CSF 2.0 Community Profile)Incident response integrated with CSF 2.0Whole playbook
NIST CSF 2.0DE.AE-02, RS.AN-03, RS.CO-02, RC.RP-013, 3 and 8, 7, 6
NIST SP 800-53 Rev. 5IR-4 Incident Handling, IR-5 Incident Monitoring, IR-6 Incident Reporting, IR-8 Incident Response Plan, SI-4 System Monitoring, AU-6 Audit Record Review3 to 8
MITRE ATLASAML.T0051, AML.T0053, AML.T0080, AML.T0085, AML.T00861, 3
OWASP Top 10 for LLM Applications 2025LLM01:2025 Prompt Injection1, 5

Metrics

  • Mean time from first detection to carrier quarantine.
  • Percentage of agents with complete trace logging and chunk-level source lineage.
  • Number of agents that read untrusted content and also hold write or send tools without approval.
  • Regression pass rate for the injection test set after each release.
  • Exercises completed per year against target.

For background on the attack class and prevention controls, see the Prompt Injection Guide and the OWASP LLM Top 10 LLM01 Prompt Injection Guide.

Standards and references

Related NHI Mgmt Group resources: AI Agent Incident Response Playbook · Prompt Injection Guide · AI Agent Goal Hijacking Guide · AI Agent Observability and Incident Response Guide

PDF for subscribers

Take this guide with you

Every section, table and checklist in one PDF, with links to the sources and related guides. Subscribe to our weekly NHI & AI Identity Journal and we’ll email you a download link.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Researched and written by the NHI Mgmt Group editorial team using AI tools, under the editorial direction of Lalit Choda. Facts are drawn from the sources listed on this page and checked by our automated quality process. Last updated 13 November 2026. Spotted an error? Use our contact page.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM and Agentic AI security. nhimg.org

PDF Downloads for Newsletter Subscribers

Get the guide

Enter your details and we’ll email you a download link.

The full guide is free to read online. The PDF is a thank you for subscribing.

Fill in all fields marked * and tick the PDF box to continue.