AI agent memory is the set of mechanisms that let an AI assistant or agent carry information beyond a single model call, either within one conversation (short-term memory) or across many conversations and sessions (long-term memory). Large language models are stateless: each request only sees what is sent in its context window, so any "memory" is something the application or provider stores and feeds back in. This page explains the main types of memory, how developer memory services and consumer app memory features work, and what controls users and administrators have. It is written for learners first, then for developers and architects building agents. Providers and products are listed alphabetically; order is never a ranking.
Last checked: 2026-11-15.
This is an independent reference page from NHI Mgmt Group based on public provider documentation; it is not sponsored or endorsed by any provider.
Key takeaways
- Models do not remember; systems do. Memory is stored outside the model and re-inserted into the context window, by your code or by the provider's service.
- Short-term memory is the current session. It is the running conversation history, kept per session or thread, and trimmed or compacted when it grows too large.
- Long-term memory is extracted and retrieved. Services such as Amazon Bedrock AgentCore Memory and Google's Memory Bank extract facts and preferences from conversations and fetch the relevant ones later.
- Consumer apps now remember by default. ChatGPT, Claude, Gemini and Microsoft Copilot all document memory features, with settings to view, delete or turn memory off and a temporary or incognito mode.
- Memory is stored data with a lifecycle. Deleting a chat does not always delete memories derived from it, so retention, deletion and access need to be designed, not assumed.
What is AI agent memory?
An LLM call takes input tokens and returns output tokens. Nothing persists inside the model between calls. To give the impression of memory, an application stores information and puts the relevant parts back into the prompt on the next call. Everything that counts as memory therefore competes for space in the context window, which the Context Windows Explained covers in detail.
Two horizons are used across the documentation:
- Short-term memory: Amazon describes it as capturing "turn-by-turn interactions within a single session", so the agent understands a follow-up such as "What about tomorrow?" after a weather question. LangGraph calls it thread-scoped memory, persisted through a checkpointer so a thread can resume.
- Long-term memory: Amazon describes it as automatically extracting and storing key insights across sessions, "including user preferences, important facts, and session summaries". LangGraph stores long-term memory as JSON documents organised by namespaces and keys, shared across threads.
What types of memory do agents use?
LangGraph's documentation borrows three categories from human memory. They are a useful vocabulary even outside LangGraph.
| Type | What it holds | Example | Typical storage |
|---|---|---|---|
| Semantic | Facts about a user or entity | "Prefers window seats"; "works in finance" | A single profile document that is updated, or a collection of individual memory records |
| Episodic | Past experiences or interactions | How a similar support case was solved last week, used as a few-shot example | Stored examples or session summaries |
| Procedural | Rules for how to behave | The agent's system prompt or learned instructions | Prompt text the agent may refine through reflection |
LangGraph notes a trade-off for semantic memory: a profile is easier to read as a whole, while a collection of individual memories is less likely to lose information when updated.
How is memory written and retrieved?
Every memory system makes the same three decisions: when to write, what to keep, and how to fetch it back.
| Decision | Options | Trade-off |
|---|---|---|
| When to write | Hot path: the agent writes during the conversation. Background: a separate process writes afterwards (LangGraph terms) | Hot path makes memories available at once but adds latency; background writing avoids latency but needs a schedule or trigger |
| What to keep | Raw events, extracted facts, summaries, user-curated entries | Raw events are complete but large; extraction is compact but can miss or distort detail |
| How to consolidate | Merge new facts with existing ones, update or delete outdated ones | Amazon describes extraction and consolidation as the two asynchronous steps of long-term memory generation |
| How to retrieve | Load everything, filter by namespace or metadata, or semantic search | Amazon's RetrieveMemoryRecords performs semantic search; loading everything is simple but costs tokens |
| How to forget | Explicit delete, expiry, reset | Anthropic's memory tool documentation suggests periodically deleting files that have not been accessed in a long time |
Memory, compaction and retrieval: how they differ
Three techniques are often confused:
- Compaction: summarises the conversation so far to keep the active context small. Anthropic describes server-side compaction as the conversation approaches the context window limit; the OpenAI Agents SDK offers
OpenAIResponsesCompactionSession, which uses the Responses API compact call. - Memory: preserves specific information that must survive beyond the session or beyond summarisation. Anthropic recommends using compaction and the memory tool together for long-running agents.
- Retrieval-augmented generation (RAG): fetches passages from a document corpus that the organisation curates, rather than from what the agent learned in conversation. See the RAG Security Guide and the Agent Memory Poisoning vs RAG Poisoning.
The sibling page Context Engineering Explained covers how these pieces are assembled into a prompt.
Which developer memory services and tools exist?
As of 2026-11-15. Listed alphabetically by provider. Only facts stated in each provider's documentation are shown.
| Provider and product | Short-term memory | Long-term memory | Where data is stored | Retrieval |
|---|---|---|---|---|
| Amazon: Bedrock AgentCore Memory | Events captured with CreateEvent, grouped by sessionId, read with ListEvents and GetEvent | Memory strategies (built-in, built-in overrides, self-managed) extract and consolidate records; none configured means no long-term records | Fully managed service; event metadata is not encrypted with a customer managed key | RetrieveMemoryRecords semantic search; ListMemoryRecords |
Anthropic: Claude memory tool (memory_20250818) | Not a session store; pairs with context editing and compaction | Claude reads and writes files under /memories using view, create, str_replace, insert, delete and rename commands | Client-side: your application executes each operation against storage you control | Claude checks /memories before starting a task and views files as needed |
| Google: Agent Platform Memory Bank | Sessions (separate service) supply events | Generates long-term user preferences and facts from sessions using generative AI models | Google Cloud; supports VPC Service Controls, CMEK and data residency at rest | Fetch memories API; IAM Conditions control access |
| LangChain: LangGraph | Thread-scoped state persisted by a checkpointer | Store of JSON documents organised by namespace and key | Your chosen checkpointer and store backend | Search across namespaces with content filters |
| OpenAI: Agents SDK sessions | Session per ID; runner prepends history and saves new items (SQLiteSession, RedisSession, SQLAlchemySession, OpenAIConversationsSession and others) | Not stated by OpenAI for the sessions feature (sessions hold conversation history) | Your database, or server-side via the OpenAI Conversations API | get_items; compaction via OpenAIResponsesCompactionSession |
Each provider places responsibility differently. With Anthropic's memory tool your handler performs every file operation, so it must validate paths, for example rejecting ../ sequences and URL-encoded traversal. The OpenAI Agents SDK notes that SQLiteSession does not verify stored rows and that session IDs select history but do not authenticate users. Amazon advises against putting sensitive content in event metadata. Google documents CMEK and IAM Conditions for Memory Bank.
How do consumer AI app memory features work?
As of 2026-11-15. Listed alphabetically by product.
| Product | What it remembers | Default | User controls | Private mode | Organisation controls |
|---|---|---|---|---|---|
| ChatGPT (OpenAI) | Saved memories and reference to chat history | Free (lighter version), Plus and Pro; OpenAI announced Team, Enterprise and Edu access for later | Settings > Personalization: turn off, manage, delete; ask it to forget | Temporary Chat: not in history, does not use or update memory, not used for training | Enterprise owners can turn memory off for the organisation |
| Claude (Anthropic) | Individual topics saved as you chat; separate memory per project; chat search | On by default for Free, Pro, Max and Ultra; Team and Enterprise owners control availability | Settings > Your Claude: view, edit, delete topics, pause or reset memory; turn memory off per chat | Incognito chats: not saved to history or memory | Owners can turn memory on or off for the organisation; owners cannot view members' memories |
| Gemini app (Google) | Key details and preferences from past chats | On by default (announced August 2025); personal Google Accounts only, not work or school accounts | Settings > Personal context > Your past chats with Gemini; manage conversations in activity settings | Temporary Chats: not used for personalisation or training, kept up to 72 hours | Not available with work, school or supervised accounts |
| Microsoft Copilot (Microsoft 365) | Saved memories, details inferred from chat history, custom instructions | Preview; Enhanced personalization control on by default | Settings > Personalization: custom instructions, saved memories, chat history | Temporary chats (accessible to admins in Purview) | Tenant control via Microsoft Graph enhancedPersonalizationSetting; eDiscovery search and deletion |
Retention detail differs and is worth reading closely. OpenAI states that deleting a chat does not delete its memories. Microsoft states the same for saved memories, stores memory in a hidden folder of the user's Exchange mailbox, and notes that Purview retention policies do not apply to Copilot memory and that memory actions do not generate Purview audit entries. Anthropic states that memory entries appear in data exports even when memory is off, and that Enterprise audit logs record owner toggle changes but not individual memory edits. Google keeps Temporary Chats for up to 72 hours to respond and process feedback.
Sensitive information in memory
Providers describe different default handling of sensitive topics, as of 2026-11-15:
- Anthropic (Claude): topics such as health, politics and religion are not stored by default; users can opt in with "Include sensitive topics in memory"; some data, such as government ID and financial account numbers, is never saved.
- Google (Gemini app): not stated in the announcement checked.
- Microsoft (Copilot): administrators cannot restrict what types of information are added to memory.
- OpenAI (ChatGPT): designed to avoid proactively remembering sensitive information, such as health details, unless the user asks.
For developer-built agents, sensitive-data rules are yours to implement. Anthropic's memory tool guidance, for example, recommends adding your own validation even though Claude usually refuses to write sensitive information.
How to design memory for an agent
- Start with short-term only. Persist the session history per user and session, with trimming or compaction, before adding long-term memory.
- Decide what is worth remembering. Write down the memory schema (preferences, facts, summaries) and what must never be stored.
- Scope by user and tenant. Use namespaces, session IDs or per-user directories, and enforce authorisation in your code; an ID that selects history is not authentication.
- Choose write timing. Hot path where users must see memories immediately; background extraction where latency matters.
- Retrieve selectively. Fetch only memories relevant to the current task to save tokens and limit exposure.
- Give users visibility. Offer view, edit and delete, as the consumer apps do, and make deletion cascade to derived memories.
- Set expiry. Remove memories that have not been used for a defined period.
- Test for poisoning and leakage. Include cross-user and injected-instruction cases in your test plan before launch.
Where security fits
Persistent memory turns a one-off prompt injection into something that can affect every later session, and a mis-scoped store can leak one user's data to another. NHI Mgmt Group covers these risks in depth: see the AI Agent Memory Security Guide for controls, the AI Agent Memory Poisoning Guide for the attack pattern, and the AI Agent Memory Poisoning Response Playbook for incident handling. The MemTensor MemoryOS supply chain attack shows that memory components are also software dependencies with their own supply chain risk.
Practitioner checklist
- Document which memory each agent uses: session history, long-term store, provider memory feature, or none.
- Scope every memory store by tenant and user, and enforce access checks in application code.
- Define a memory schema and a list of data that must never be written, and validate writes against it.
- Use compaction for long sessions and long-term memory only for information that must persist.
- Provide view, edit and delete for users, and confirm deleting a chat also handles memories derived from it.
- Set expiry for unused memories and include memory stores in retention and data subject request processes.
- For consumer AI apps in the organisation, review the admin memory setting and the temporary or incognito chat policy.
- Add memory poisoning and cross-user leakage tests to agent release testing.
Standards and references
- Amazon: AgentCore Memory, Memory types and Memory strategies (accessed 2026-11-15)
- Anthropic: Memory tool (accessed 2026-11-15)
- Anthropic: Chat search and memory in Claude (accessed 2026-11-15)
- Google Cloud: Memory Bank overview (accessed 2026-11-15)
- Google: Temporary Chats and personalisation in the Gemini app and Gemini Apps personalisation help (accessed 2026-11-15)
- LangChain: LangGraph memory overview (accessed 2026-11-15)
- Microsoft: Copilot personalization and memory (accessed 2026-11-15)
- OpenAI: Memory and new controls for ChatGPT and OpenAI Agents SDK: Sessions (accessed 2026-11-15)
Related NHI Mgmt Group resources: AI Agent Memory Security Guide · Context Windows Explained · Context Engineering Explained · AWS Bedrock AgentCore Security Guide · Vertex AI Agent Engine security guide