LLM memory is the structured state an agent keeps across interactions so it can remember preferences, tasks, and prior outcomes. Unlike a simple chat log, it can influence future decisions, which makes retention, access, and deletion part of the security design, not just the user experience.
Expanded Definition
LLM memory is the persistence layer that allows a Large Language Model based system, especially an agent, to carry forward context from one interaction to the next. It may store user preferences, task state, summaries, tool outputs, or prior decisions, but it should not be treated as a neutral transcript. Because memory can shape future behaviour, it becomes part of the system’s security boundary and governance model.
Definitions vary across vendors and implementations. Some products use short-lived session memory only, while others maintain long-term memory, retrieval stores, or user-scoped profiles. The security significance is not the storage medium alone, but whether the retained state can influence tool use, access decisions, or content generation. That is why memory handling aligns closely with the governance expectations described in the NIST AI Risk Management Framework and the NIST AI 600-1 Generative AI Profile, both of which emphasise controllable risk, transparency, and lifecycle oversight.
The most common misapplication is equating memory with a harmless chat history, which occurs when teams persist prompts and outputs without defining who can read, modify, expire, or delete the stored state.
Examples and Use Cases
Implementing LLM memory rigorously often introduces retention and access-control complexity, requiring organisations to weigh personalisation and continuity against privacy, prompt-injection exposure, and data minimisation obligations.
- An internal support agent remembers a user’s preferred language and escalation path, reducing repeated context gathering while increasing the need for scoped access and deletion rules.
- A coding assistant stores prior repository decisions so it can maintain architectural consistency, but the stored notes must be protected from poisoning and cross-project leakage.
- An enterprise agent retains task state for a procurement workflow, so later steps can continue after a pause; that memory must be auditable because it can change tool invocation behaviour.
- A customer-facing assistant keeps a long-term profile of preferences. If the profile includes personal data, retention and erasure controls should align with the NIST AI 600-1 Generative AI Profile and privacy governance expectations.
- An autonomous agent caches prior security decisions and remediation outcomes. In practice, that memory can help with continuity, but it also creates a higher-value target for manipulation, which is why the OWASP Agentic AI Top 10 is relevant to threat modeling.
Why It Matters for Security Teams
LLM memory changes the blast radius of a model from a single response to an ongoing decision substrate. If memory is poisoned, over-retained, or exposed across users, the system can repeat unsafe instructions, reveal sensitive data, or make future actions based on corrupted state. Security teams therefore need controls for provenance, TTLs, segmentation, review, and deletion, not just model output filtering. This is especially important for agentic systems where memory can influence tool calls, retrieval, and autonomous follow-on actions.
The identity dimension is also material. When memory is scoped to a user, role, service account, or Non-Human Identity, the system should treat that context as an access-bearing asset rather than a convenience feature. That is why the operational conversation often overlaps with the OWASP Top 10 for Agentic Applications 2026 and, for adversarial manipulation scenarios, the MITRE ATLAS adversarial AI threat matrix.
Organisations typically encounter the consequences of memory misuse only after an agent repeats a stale decision, exposes a previous user’s context, or executes a tool action based on poisoned state, at which point memory governance becomes operationally unavoidable to address.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, OWASP Non-Human Identity Top 10 and MITRE ATLAS address the attack and risk surface, while NIST AI RMF and NIST AI 600-1 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | AI RMF governs risk, transparency, and lifecycle oversight for memory-bearing AI systems. | |
| NIST AI 600-1 | Generative AI profile addresses lifecycle controls relevant to retained state and context. | |
| OWASP Agentic AI Top 10 | Agentic AI guidance covers memory abuse, poisoning, and unsafe state carryover. | |
| OWASP Non-Human Identity Top 10 | NHI guidance is relevant when memory is tied to service identities or agent accounts. | |
| MITRE ATLAS | ATLAS documents adversarial AI techniques that can target retained state. |
Define memory ownership, review, and deletion controls within your AI risk management process.
