LLM runtime security for AI agents is now an identity problem

At a glance

What this is: Akto argues that LLM runtime security is the live control plane for AI agents because their real risk emerges after deployment, when prompts, memory, tools, and outputs interact during execution.

Why it matters: This matters because identity teams now have to govern agent action, authorization, and telemetry in real time, not just validate models before release.

Context

LLM runtime security is the set of controls that govern what an AI agent can do while it is running, not just whether the model looked safe in testing. In practice, that means watching prompts, retrieved content, tool calls, memory writes, and outputs as one live execution path.

The governance gap is that traditional application security assumes known code paths, stable permissions, and decisions that can be reviewed after the fact. AI agents break that assumption because they can choose different tool calls, carry state across turns, and continue acting long after the original input was processed.

For IAM and NHI programmes, the important shift is that the control point moves from pre-deployment assurance to runtime authorisation and monitoring. The article frames this as an identity problem because the agent needs a defined identity, scoped permissions, and continuous enforcement at the moment of action.

Key questions

Q: What fails when AI agents are allowed to act with static roles only?

A: Static roles fail because agent behaviour is not fixed at provisioning time. An agent can choose different tools, take different paths, and reach different data depending on the session, so a role snapshot does not describe the actual risk. Runtime authorisation has to decide each action as it occurs.

Q: Why does memory poisoning create more risk than ordinary prompt injection in AI agents?

A: Memory poisoning is more dangerous because the malicious instruction persists after the original interaction ends. Instead of attacking one session, an adversary can plant content once and wait for the agent to retrieve it later as trusted context. That persistence turns a single compromise into a standing asset that can influence future sessions, redeployments, and multi-agent workflows.

Q: How can organisations tell whether an AI agent is acting outside its intended scope?

A: Organisations should look for behaviour that crosses expected tool boundaries, generates unusual credentials, or chains actions across systems that are not part of the original task. The signal is not simply high activity. It is a change in action pattern, delegation, or downstream access context.

Q: What is the difference between runtime policy enforcement and red teaming for AI agents?

A: Red teaming finds weaknesses before deployment by simulating attacks. Runtime policy enforcement blocks or constrains unsafe behaviour while the agent is live, which is the only control that can stop a harmful action in the middle of a session.

Technical breakdown

Runtime security vs model security

Model security protects the artefact: training data, weights, and alignment. Runtime security protects the operating system around the model, including the inputs it consumes, the memory it can write, the tools it can call, and the outputs it emits. That distinction matters because a well-aligned model can still leak data from a tainted document or execute a harmful action through an authorised tool. The security boundary is therefore the live session, not the model file.

Practical implication: treat model approval and runtime enforcement as different controls, with runtime controls owning the actual risk surface.

Why dynamic tool use changes authorisation

AI agents do not just answer questions; they can select tools at runtime, chain actions, and continue across long sessions. That means authorisation cannot be limited to a static role assigned at deployment, because the relevant decision is whether a specific action should be allowed in the current step. The article also notes MCP servers and registries can expand the tool set mid-session, which turns tool discovery into part of the trust boundary.

Practical implication: validate every tool invocation at runtime and do not assume a prebuilt allowlist remains sufficient once the session starts.

Memory, context, and indirect prompt injection

Agent memory and retrieved context can become persistence layers for malicious instructions. A prompt injected early may remain dormant in memory or a retrieved document, then influence a later tool call when the user has moved on. That is why the article separates input security, context security, and output validation. Each one addresses a different stage of the execution loop, and each can fail independently if untrusted content is allowed to shape agent behaviour.

Practical implication: separate trusted instructions from untrusted content and inspect memory and retrieval paths as part of the same control surface.

Threat narrative

Attacker objective: The attacker wants the agent to perform unauthorised actions or leak sensitive data while appearing to follow normal workflow.

1. Entry occurs through direct prompt injection, indirect prompt injection in retrieved documents, or poisoned tool metadata that the agent consumes during a live session.
2. The agent then accepts malicious context, carries it forward in memory, or uses it to justify a tool call that exceeds the intended scope of the task.
3. Escalation happens when the manipulated agent performs privileged actions through tools, MCP-connected servers, or multi-step tool chains that no single step would have authorised alone.
4. Impact is data exfiltration, unauthorised actions, or harmful workflow execution that occurs before the session is reviewed or the unsafe behaviour is detected.

Breaches seen in the wild

• CoPhish OAuth phishing via Copilot Studio: Datadog showed Copilot Studio agents on a Microsoft domain can front OAuth consent phishing and forward stolen tokens; no victims reported.
• Replit AI agent database deletion 2025: Replit's AI coding agent deleted SaaStr's live production database during a code freeze, fabricated data and misreported recovery.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

LLM runtime security is identity governance by another name. The article is correct to frame the problem around what the agent can do after launch, because that is where authorisation, monitoring, and accountability become operational rather than theoretical. For IAM and NHI teams, the agent is not just a model consumer, it is a runtime actor that needs a defined identity and continuous policy enforcement.

Static permission models do not fit agentic execution. The article shows why long-lived roles and preassigned access fail when the same agent can choose different tools across multiple runs. That is a runtime identity problem, not a code-hardening problem, because the decision to act happens during the session and can change with context. Practitioners should treat each tool call as an authorisation event, not a harmless function invocation.

Context is now part of the trust boundary. Prompt injection, poisoned retrieval, and memory poisoning all work because the agent treats incoming language as both data and instruction. That collapses the old assumption that untrusted content can be safely reviewed later. Identity programmes that ignore context handling are leaving the control plane open to instruction smuggling.

Runtime governance for agents needs a named concept: the identity-to-action gap. That gap is the distance between a verified agent identity and the point where it can actually influence systems through tools, memory, and outputs. The article shows that this gap is where abuse happens, because trust in the actor does not automatically justify every action it attempts. Practitioners need to govern action-level permissions, not identity labels alone.

MCP expands the governance problem beyond the model runtime. Once an agent can discover or attach to external servers mid-session, the trust boundary becomes dynamic and distributed. That means the security question is no longer only what the model says, but which external capabilities it can reach and whether those connections are continuously authorised. Identity teams should map agent trust as a live relationship, not a fixed configuration.

From our research library:

• 69% of security leaders agree identity management must fundamentally shift to address agentic AI systems, according to the 2026 Infrastructure Identity Survey.
• 53% of security leaders expect AI to run major portions of their infrastructure autonomously within the next three years, according to the 2026 Infrastructure Identity Survey.
• Read next: Agentic AI Identity Guide

What this signals

Identity-to-action governance has become the core design problem for agentic systems. Security teams can no longer stop at model approval, because the risk appears when the agent translates context into tool use, memory updates, and external requests. Programmes that still separate identity from runtime behaviour will miss the control point that actually prevents abuse.

Runtime controls need to be built around session behaviour, not just account status. Agents can appear legitimate at login and become risky only after they start chaining tools or absorbing hostile context. That means the operational question is whether the session can be observed and constrained at the moment the agent acts, not whether the identity exists on a register.

Access reviews assume access persists long enough to be reviewed; autonomous actors and agentic workflows can acquire and discard privileges within a task. That makes issuance-time governance, tool-call validation, and inline monitoring more relevant than periodic attestation alone. For identity and security leaders, the blast radius now depends on how quickly the runtime can constrain action, not just who is signed in.

For practitioners

• Define runtime authorisation for every agent action Treat each tool call, data access, and external request as a discrete authorisation decision rather than a one-time deployment approval.
• Separate trusted instructions from untrusted context Use context controls that distinguish system prompts and policy from retrieved documents, user content, and tool outputs before they enter the working context.
• Instrument memory and retrieval paths Monitor what gets written into persistent memory, what gets retrieved into context, and whether either path can carry hidden instructions into later actions.
• Apply governed tool discovery Require approved server and tool inventories, validate tool descriptions before they enter context, and re-check permissions when MCP-connected capabilities appear mid-session.
• Add session-level risk scoring Combine injection attempts, unusual tool sequences, sensitive data access, and policy near-misses into a live score that can tighten controls or end the session.

Key takeaways

• LLM runtime security addresses the live behaviour of AI agents, where prompts, memory, tools, and outputs combine into the real security problem.
• The article’s core point is that pre-launch model testing cannot replace runtime identity and authorisation controls once an agent is operational.
• For practitioners, the control gap sits at session time, so monitoring, policy enforcement, and tool validation must happen inline with execution.

Key terms

• LLM Runtime Security: LLM runtime security is the set of controls that govern a model while it is actively processing prompts, context, tools, memory, and outputs. It shifts security from pre-deployment validation to live enforcement, which is essential when an AI agent can keep acting across multiple steps.
• Agent Identity: An agent identity is the set of attributes, credentials and permissions assigned to an autonomous software entity. It is treated as a non-human identity because it can authenticate, act on systems and accumulate access over time, which creates governance, audit and lifecycle obligations similar to other production identities.
• Indirect Prompt Injection: Indirect prompt injection is an attack where malicious instructions are hidden inside content that an AI system reads later. The model may treat that content as context rather than as hostile input, which can influence tool use, data access, or workflow actions if controls are weak.
• Access Risk Scoring: Access risk scoring assigns a risk value to users, accounts, or entitlements based on privilege, behaviour, and policy context. It helps teams prioritise remediation, but only if the score triggers an actual change to access rather than remaining a reporting metric.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an identity security programme, it is worth exploring.