Short-Term Memory

Short-term memory is the working context an AI uses during a live conversation. It usually combines recent messages with a summary of older exchanges so the model can stay coherent without carrying the entire transcript. This layer is temporary, continuously updated, and optimized for immediate response quality.

Expanded Definition

Short-term memory in AI is the active, session-level context that supports coherent responses during an ongoing interaction. It is not permanent storage, and it is not the model itself. Instead, it is the working set of recent prompts, prior turns, tool outputs, and sometimes compressed summaries that help the system preserve relevance across a conversation. In practice, this memory layer is often managed by the application, orchestration stack, or agent framework rather than by the NIST Cybersecurity Framework 2.0 directly.

Definitions vary across vendors, especially where “memory” is used to describe caching, conversation history, retrieval, and long-term user profiles in the same product. For glossary purposes, short-term memory should be understood as transient context that can be refreshed, truncated, summarised, or lost when a session ends. In agentic systems, it may also include scratchpad state or tool results that shape the next action without becoming durable knowledge.

The most common misapplication is treating short-term memory as a secure record of truth, which occurs when teams assume every prior turn remains available, accurate, and safely retained across sessions.

Examples and Use Cases

Implementing short-term memory rigorously often introduces a context-management tradeoff, requiring organisations to weigh smoother conversational continuity against token limits, latency, and the risk of carrying forward stale or sensitive data.

• A support chatbot keeps the last few user messages in context so it can answer follow-up questions without asking the customer to repeat details.
• An AI agent uses a compact summary of prior actions before deciding whether to call a tool again, which helps preserve workflow continuity.
• A copilot in a security operations workflow maintains the active incident thread, including alerts, analyst comments, and recent enrichment results, while dropping older unrelated exchanges.
• A retrieval-augmented generation system stores only the immediate interaction state in short-term memory, while external knowledge remains in the retrieval layer rather than the conversation buffer.
• A workflow assistant that handles identity tasks, such as access review or password reset guidance, uses ephemeral context to remember the current request, but should not treat that state as an authoritative identity source.

For organisations designing memory behavior, the NIST Cybersecurity Framework 2.0 is useful as a governance reference for managing data exposure, access, and operational risk around transient AI context.

Why It Matters for Security Teams

Short-term memory becomes a security concern when transient context includes secrets, personal data, incident details, or instructions that should not persist beyond the active session. If that context is over-retained, under-sanitised, or exposed to the wrong tool call, an AI system can repeat sensitive information, make unsafe decisions, or leak operational details across users. If it is too aggressively shortened, the system may lose task continuity and force operators into repetitive re-entry, which increases error rates and user workarounds.

For identity and NHI-adjacent workflows, this matters because an AI assistant may hold temporary access-request context, approval state, or agent instructions that shape privileged actions. That ephemeral state should be treated as high-risk operational data, especially when an agent has execution authority or access to credentials and tokens. Security teams should define what belongs in session memory, what must be redacted, and what must never be written into prompts or summaries. Guidance from the NIST Cybersecurity Framework 2.0 maps well to these controls because it emphasizes governance, access discipline, and resilience around sensitive operational processes.

Organisations typically encounter the consequences only after an AI system repeats a stale instruction, exposes a previous user’s context, or acts on an outdated session summary, at which point short-term memory becomes operationally unavoidable to address.

Related resources from NHI Mgmt Group

• When does a short-lived credential still become a long-term risk?
• Why do short-lived cloud permissions still create long-term risk?
• Why do AI agents with long-term memory create more security risk than stateless chatbots?
• Why do memory scrapers remain effective even when secrets are short-lived?