Memory is missing in AI systems: what shipped this week

At a glance

What this is: This is Arize’s taxonomy of the AI memory features shipping right now, and its central finding is that most of them are not memory in the human sense.

Why it matters: It matters to IAM, NHI, and AI security practitioners because any system that persists context across sessions, rewrites state, or retrieves from governed stores creates new identity, access, and audit questions.

By the numbers:

• HydraDB shipped a memory layer with a 90.79% LongMemEval-S claim.
• Anthropic’s compact-2026-01-12 beta header triggers at 150K tokens by default.
• 28-day TTL.

👉 Read Arize’s analysis of AI memory architectures and what is actually shipping

Context

AI memory is now a governance problem as much as an engineering one, because systems that retain, rewrite, or retrieve user and organisational context create durable state that can outlive the session that produced it. The primary issue is not whether a model can store something, but whether the system can preserve truth, scope, and access control as that state is reused.

For identity and security teams, the relevant question is who can write memory, who can read it, and what happens when the stored context becomes stale, contradictory, or overbroad. That is where retrieval, session compaction, and cross-session consolidation intersect with NHI governance, workload identity, and auditability.

Arize’s catalogue is most useful because it separates marketing language from architecture. The systems it describes are typical of where the market is heading, not exceptions.

Key questions

Q: How should security teams govern memory in AI agents?

A: Treat memory as a privileged state store, not a convenience feature. Define what can be retained, who owns the decision, how long it persists, and which identity scope it belongs to. Then test whether stored context can influence later actions outside the original session. If it can, the memory layer needs the same governance discipline applied to other high-trust systems.

Q: Why does summarised agent memory drift over time?

A: Summarised memory drifts because each compression step rewrites prior context into a new approximation. Small omissions and paraphrases accumulate, so later sessions rely on an increasingly distorted version of the original conversation. The risk is higher when summaries are repeatedly updated without retaining immutable source history for verification.

Q: How do security teams know if AI memory controls are working?

A: They know the controls are working when they can prove which entries were written, why they were accepted, who can reuse them, and whether retrieved context is checked before action. If the team cannot trace those steps end to end, memory governance is still incomplete and the blast radius remains unclear.

Q: What is the difference between retrieval memory and cross-session consolidation?

A: Retrieval memory fetches relevant stored context at runtime, while cross-session consolidation rewrites or reorganises past context into a new persistent artifact. Retrieval mostly affects what gets surfaced; consolidation changes what the system will remember next time. The second carries more truth-drift risk and demands stronger version control.

Technical breakdown

Retrieval-based memory and semantic indexing

The first pattern is retrieval augmented memory, where data is chunked, embedded, and stored in a vector database, sometimes with a graph layer for relationships. At inference time, a query embedding retrieves the top-k matches, which makes the system better at finding relevant context but not necessarily at preserving truth across time. The benchmark numbers often describe retrieval recall, not end-to-end task correctness, so they can overstate what the memory layer actually delivers. This is closer to semantic search than durable memory. Practical implication: treat retrieval as a relevance mechanism, not as proof that the system remembers correctly.

Practical implication: Measure whether retrieved context still answers the user’s question correctly, not whether the index found a nearby document.

Compaction, summarisation, and context drift

The second pattern compresses long conversations into shorter working context so agents can keep operating within token limits. This can be done by LLM summaries or by structural indexing that preserves references without rewriting the source. The security and reliability issue is that iterative summaries can drift, because each compression step becomes a new sample of the prior state. Once enough passes accumulate, small factual errors become baked into the working memory. Structural compaction avoids that specific failure mode because it does not rely on an LLM to decide what survives. Practical implication: preserve raw session history alongside summaries so compressed state can be audited and reconstructed.

Practical implication: Keep immutable source context and compare summaries against it before trusting downstream agent behaviour.

Cross-session consolidation and state rewrite risk

Cross-session memory tries to preserve useful facts across interactions by reorganising stored context into a new memory object. The key architectural difference is whether the system writes in place or produces a parallel artifact, because in-place rewriting is more exposed to consolidation error and truth drift. The article’s comparison between two lab implementations matters because it shows that ‘memory’ can mean very different write semantics. A preserved input store gives you rollback and comparison options, while an overwritten store hides what changed. In security terms, write semantics are part of the control plane. Practical implication: version memory state and require traceable diffs before consolidation is promoted into production.

Practical implication: Separate input and output stores so memory changes remain reviewable and reversible.

NHI Mgmt Group analysis

Memory is becoming an identity surface, not just a product feature. Once an agent can retain user context across sessions, the system is no longer stateless automation. That means write access to memory becomes as sensitive as read access to secrets or tokens, especially when memory can influence future decisions. For IAM and NHI programmes, the governance question is who can mutate the retained state and whether that state is lifecycle-managed like any other privileged asset.

Retrieval-layer memory creates a trust boundary problem that most teams are underestimating. A vector index or semantic cache may look like a convenience layer, but it becomes part of the decision path once the agent treats retrieved context as authoritative. If stale or poisoned context can be surfaced without provenance, the organisation has effectively outsourced memory integrity to a search mechanism. Practitioners should map this to NIST CSF and access-control thinking, not just model performance.

In-place consolidation is the named failure mode here: state rewrite without an auditable original. The article’s strongest warning is that rewritten memory can drift from the source of truth and hide the difference. That is a governance gap, not a tuning issue. Teams should treat memory consolidation as a controlled change process with lineage, rollback, and review, especially where agent output can affect customer actions, policy decisions, or downstream workflow execution.

Filesystem-shaped agent memory is a pragmatic control pattern, but it shifts responsibility to storage governance. When the model is the index and the filesystem or object store holds the maintained knowledge base, access controls, retention rules, and offboarding become the real security control points. That is where NHI governance intersects directly with agentic AI. The practical conclusion is that storage permissions now govern agent capability as much as prompts do.

The market is converging on workable memory, not solved memory. Most shipped systems optimise one slice of the problem, such as retrieval relevance or working-context preservation, while the genuinely hard part remains multi-evidence reasoning across changing facts. Security and architecture teams should expect more stateful AI products, but they should not confuse persistence with correctness. The field still needs explicit governance for what the memory layer may remember, rewrite, and expose.

What this signals

Memory systems will force security teams to think in terms of state governance, not only session security. Once an AI system can preserve context across interactions, every retained artifact becomes part of the control surface. Teams should expect demand for provenance, retention, and rollback controls to rise alongside agent adoption, especially where memory can shape customer or operational outcomes.

Persistent memory also sharpens the need for NHI-style lifecycle discipline. When services and agents can write to long-lived state, the problem looks less like prompt management and more like privileged access to a governed store. That is where the control model aligns with NHI Lifecycle Management Guide thinking and, where relevant, with NIST Cybersecurity Framework 2.0 governance expectations.

Memory drift is now an operational risk signal. If consolidation or summarisation changes the facts an agent relies on, the issue is not model quality alone but broken state lineage. Teams should prepare to test memory artifacts the way they test other security-critical data paths: for integrity, provenance, and revocation readiness.

For practitioners

• Classify memory as governed state Map every persistent AI memory store to an owner, purpose, retention period, and access model. Treat writes to memory as privileged actions that require the same lifecycle discipline as any other sensitive system state.
• Separate source history from consolidated memory Keep immutable session transcripts alongside any summarised or consolidated memory artifact so teams can compare versions, investigate drift, and roll back bad consolidation decisions.
• Limit write access to memory artifacts Restrict which services, agents, or operators can modify long-lived memory. Use short-lived credentials and explicit service identities for memory writes, especially where the memory influences customer-facing or policy-bearing actions.
• Audit memory for provenance and supersession Record where each remembered fact came from, when it was last confirmed, and whether it has been superseded. That metadata is essential for detecting stale context and preventing agents from acting on obsolete state.

Key takeaways

• The article’s central warning is that most shipping AI “memory” is still a narrower mechanism than the word implies.
• The risk is not just wrong recall, but state drift, hidden rewrites, and ungoverned persistence across sessions.
• Teams should govern memory like privileged state, with provenance, lifecycle control, and auditable change history.

Key terms

• Retrieval Augmented Memory: A memory pattern that stores content externally and fetches relevant pieces at runtime using embeddings or other search methods. It improves recall and relevance, but it does not automatically preserve truth, supersession, or auditability across sessions.
• Cross-Session Consolidation: A process that rewrites or reorganises past interactions into a new persistent memory artifact for later use. It can improve continuity, but it also introduces drift risk because the system may change the remembered state rather than merely reference it.
• Memory Provenance: The evidence trail showing where a remembered fact came from, when it was last confirmed, and whether it has been replaced. Provenance is essential when AI systems reuse state, because without it teams cannot reliably audit or correct memory-driven decisions.
• Context Drift: Context drift is the gap between what an identity was authorised to do at the start of a session and what it ends up doing after inputs, tools, or instructions change. In agentic systems, it is a core governance problem because behaviour can move outside the original approval boundary.

What's in the full article

Arize’s full article covers the architectural detail this post intentionally leaves at the classification level:

• The four-bucket taxonomy with product-by-product placement and the reasoning behind each classification
• The benchmark and mechanism discussion behind LongMemEval, LoCoMo, ConvoMem, and MemBench comparisons
• The contrast between preserved memory stores and in-place rewriting in cross-session systems
• The specific engineering patterns behind filesystem-shaped memory, including LargeJson and PlanMessage

👉 Arize’s full article covers the benchmark details, product examples, and architectural comparisons behind each memory bucket.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management in a way that helps teams apply identity controls to persistent AI systems. It is designed for practitioners who need to connect identity lifecycle thinking to modern AI and automation risk.