AI Agent Memory Poisoning Guide: Attacks, Detection and Defences

Memory poisoning turns a single piece of malicious content into a lasting change in how an AI agent reasons, because whatever the agent stores about a user, a project or a task is fed back into every later session as trusted context. Prompt injection ends when the conversation ends; poisoned memory does not. This guide explains how memory and context poisoning works across assistant memory, project instruction files, session summaries, shared agent memory and RAG stores, how to detect it and how to harden agent memory so a bad write can be found, quarantined and rolled back. It is written for security architects, AppSec and platform teams who build or approve agents with persistent memory. It deliberately stays defensive: no payloads or attacker procedures.

Key takeaways

• Persistence is the difference. OWASP ASI06 Memory and Context Poisoning covers stored and retrievable context; a one-off injected prompt is LLM01:2025 Prompt Injection.
• Every memory write is an input channel. Assistants, summarisers and coding agents write memory from content they read, so a web page, document or tool result can become a standing instruction.
• Memory is shared more often than teams think. Project files, team memory, multi-agent stores and vector indexes let one poisoned record affect other users and other agents.
• Treat memory as data with provenance. Record who or what wrote each entry, from which source, and gate writes that change instructions or trust.
• Plan for rollback before you need it. Versioned memory, snapshots and a quarantine state are what make recovery possible without wiping everything.

How the attack works

Agents carry knowledge between sessions in several ways: explicit saved memories ("remember that I prefer..."), automatic summaries of earlier sessions, instruction files checked into a repository, long-running threads and retrieval stores. OWASP describes the risk as attackers corrupting or seeding this context so that later reasoning, planning or tool use becomes biased, unsafe or prone to exfiltration. MITRE ATLAS models it as AML.T0080 AI Agent Context Poisoning, with sub-techniques AML.T0080.000 Memory (persisting changes into future sessions) and AML.T0080.001 Thread (instructions that persist for the rest of a long thread).

The attack has three stages that defenders can break:

1. Delivery: untrusted content reaches the agent, usually through indirect prompt injection (AML.T0051.001) in a web page, shared document, email, tool output or a link that pre-fills a prompt. See the Prompt Injection Guide for the delivery mechanics, which this guide does not repeat.
2. Write: the agent, or a background process such as a session summariser, stores something derived from that content: a false fact, a preference, a "trusted source", or an instruction. The write often looks like normal behaviour, because remembering things is the feature.
3. Recall: in a later session, possibly for a different user or a different agent, the stored entry is loaded into context with more authority than the original content had. It then shapes answers or triggers tool calls, for example sending data out through a web or email tool (AML.T0086 Exfiltration via AI Agent Tool Invocation).

Research shows the write does not need privileged access. The MINJA paper (Dong et al., 2025) describes planting malicious records in an agent's memory bank purely through normal queries, designed to be retrieved as worked examples when a different user later asks a related question. OWASP also flags "bootstrap poisoning", where an agent re-ingests its own earlier outputs into trusted memory, so one bad answer becomes a durable fact.

Where it shows up

Project memory deserves particular attention in engineering teams. The Claude Code documentation is explicit that memory files are treated "as context, not enforced configuration", and that a hook, not an instruction, is the way to block an action. A pull request that edits an agent instruction file is therefore a change to agent behaviour and should be reviewed like one. See the AI Coding Agents Security Guide.

Not all memory poisoning is criminal. Microsoft's Defender research team (February 2026) described "AI Recommendation Poisoning": "Summarize with AI" buttons and links that pre-fill a prompt asking the assistant to remember a company as a trusted source. Over 60 days of email traffic it found 50 distinct prompts from 31 real companies. The mechanism is identical to a malicious attack, which is why the controls below apply regardless of intent.

Impact

• Persistent exfiltration: Johann Rehberger's 2024 research showed instructions planted in ChatGPT memory causing ongoing leakage of later conversations through image rendering. OpenAI fixed the exfiltration channel in ChatGPT 1.2024.247 (September 2024), but the researcher notes that writing memories through prompt injection was not itself fixed.
• Corrupted decisions: a false fact stored as truth (OWASP's travel booking scenario, a fake flight price approved at payment) or a security agent whose memory has learned to label malicious activity as normal.
• Biased recommendations: injected "trusted source" memories skew advice on health, finance or security, as Microsoft observed.
• Cross-user and cross-tenant reach: shared memory and loose vector namespace filters let one record affect many users. OWASP lists cross-tenant vector bleed as a scenario.
• Goal drift: OWASP notes that memory poisoning often leads to goal hijacking. See the AI Agent Goal Hijacking Guide.
• Anti-forensics: ATLAS AML.T0092 Manipulate User LLM Chat History describes editing or deleting history to hide persistent changes, which weakens investigations that rely on the chat transcript.

Detection

Memory poisoning is detectable because writes are discrete events. Most teams fail to see it because they log prompts and responses but not memory operations.

Metrics that show the controls are working:

• Percentage of memory entries with recorded provenance (target: all entries in production agents).
• Number of memory writes blocked or quarantined per week, and the share confirmed as true positives.
• Mean time to identify and roll back a poisoned entry in a test exercise.
• Percentage of agent instruction file changes merged with a required reviewer.
• Age distribution of unverified memory (unverified entries should decay or expire).

Feed these signals into the same pipeline as other agent telemetry. See the AI Agent Behaviour Baselining and Anomaly Detection Guide.

Prevention and hardening

ATLAS mitigation AML.M0031 Memory Hardening summarises the approach: trust boundaries for how an agent stores and accesses memory, external authentication and validation for memory updates, semantic integrity checks on retrieved memories before action, and monitoring and remediation for poisoned memory. In practice:

Control the write path

• Gate writes by source: do not let content that arrived from an untrusted channel (web, inbound email, third-party tool output) write instruction-like or trust-changing memory without confirmation. OWASP recommends allowing only authenticated, curated sources.
• Scan before commit: run new memory writes through a classifier for injected instructions and sensitive data before they are stored, not only at recall.
• Separate facts from instructions: store user preferences and facts as structured fields; keep behavioural instructions in configuration that only owners can change.
• Stop bootstrap poisoning: do not automatically re-ingest the agent's own outputs into trusted memory.

Contain the blast radius

• Segment memory by user, session, tenant and domain, with per-tenant namespaces in vector stores and permission-aware retrieval. See the Permission-Aware RAG Guide.
• Weight retrieval by trust: OWASP suggests trust scores and, for high-impact memory, two factors such as a provenance score plus a human-verified tag.
• Restrict tools after recall: apply AML.M0030 Restrict AI Agent Tool Invocation on Untrusted Data when recalled memory came from untrusted sources, and require approval for consequential actions (AML.M0029).
• Use deterministic blocks: egress allowlists for agent web tools and hooks that block commands regardless of what memory says. Unit 42 lists URL allowlists and Bedrock Guardrails with the prompt attack policy as the mitigations for its proof of concept.

Make recovery possible

• Version and snapshot memory so entries can be diffed and rolled back to a validated state (OWASP Playbook 2).
• Support quarantine: a flagged entry should stop being recalled without being deleted, so it can be investigated.
• Expire unverified memory and set retention by data sensitivity.
• Give users a review surface: both Rehberger and Microsoft advise users to review saved memories and delete entries they do not recognise.

Rollout

1. Inventory: list every agent with persistent memory, what it stores, who and what can write, and whether stores are shared.
2. Log first: turn on memory write logging with source attribution in monitor-only mode for two to four weeks to baseline normal writes.
3. Pilot gating: enable write scanning and untrusted-source confirmation on one agent; measure false positives against the baseline.
4. Test: run controlled red-team exercises with benign canary instructions in documents and confirm they are blocked, quarantined or detected. See the AI Agent Security Testing Checklist.
5. Extend: apply segmentation, trust weighting and repository review rules to remaining agents, starting with shared and multi-agent stores.
6. Rollback path: keep gating behind a configuration flag so it can be relaxed per agent if it breaks workflows, and rehearse restoring memory from a snapshot.

Small vs large organisations: a small team using commercial assistants may have no write-path controls available; the practical controls are disabling memory for sensitive roles, using temporary chats for untrusted content, and periodic memory review. Organisations that build agents should own the memory service and implement provenance, gating and rollback directly.

Framework mapping

For the wider category, see the OWASP Top 10 for Agentic Applications Guide and the MITRE ATLAS Guide.

Lessons from incidents

No breach in the NHI Mgmt Group database yet involves confirmed exploitation of agent memory in the wild; the public record is research disclosures. Three lessons stand out:

• Fixing the exit is not fixing the write. In the ChatGPT case the exfiltration channel was closed, but memory could still be written from untrusted content. Treat write controls and egress controls as separate requirements.
• Platform features must be switched on. Unit 42 states that its Bedrock proof of concept did not exploit a vulnerability in the Amazon Bedrock platform, and that the prompt attack guardrail would have mitigated it. Defaults matter; check them.
• Single-shot injection incidents show the delivery route. Incidents such as EchoLeak involved instructions in retrieved email rather than stored memory, but the same retrieved content could be written to memory by an agent that summarises or remembers.

Practitioner checklist

• Inventory every agent memory surface: saved memories, summaries, instruction files, shared stores, vector indexes.
• Log every memory write with the source channel, triggering content and identity.
• Block or require confirmation for instruction-like or trust-changing writes from untrusted sources.
• Store behavioural instructions in owner-controlled configuration, not user-writable memory.
• Segment memory and vector namespaces by user, tenant and domain.
• Require code review for agent instruction files such as CLAUDE.md and AGENTS.md.
• Version memory, support quarantine and rehearse rollback from snapshot.
• Hunt for pre-filled assistant links with memory keywords in email and proxy logs.
• Expire unverified memory and give users a way to review and delete entries.
• Red-team memory writes with benign canaries before each major agent release.

Standards and references

• OWASP: Top 10 for Agentic Applications for 2026
• OWASP: Agentic AI Threats and Mitigations
• MITRE ATLAS: AI Agent Context Poisoning
• OWASP: Top 10 for LLM Applications 2025
• Palo Alto Networks Unit 42: Indirect Prompt Injection Poisoning AI Long-Term Memory in Amazon Bedrock Agents
• Microsoft Security: AI Recommendation Poisoning
• Embrace the Red: Persistent Data Exfiltration via ChatGPT Memory
• arXiv: Memory Injection Attacks on LLM Agents via Query-Only Interaction

Related NHI Mgmt Group resources: AI Agent Memory Security Guide · AI Agent Threat Modelling Guide · RAG Security Guide · Multi-Agent and A2A Security Guide · Agentic AI Glossary