AI runtime security is exposing a gap posture tools miss

At a glance

What this is: This is an independent analysis of why AI security cannot stop at posture, with runtime visibility and action analysis emerging as the key gap.

Why it matters: It matters because IAM, PAM, and cloud teams need to govern AI systems as active non-human identities whose behaviour changes after deployment, not just as static assets.

By the numbers:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%).
• 53% of MCP servers expose credentials through hard-coded values in configuration files.

👉 Read Orca Security's analysis of runtime AI security and posture gaps

Context

AI runtime security is the discipline of detecting what models, agents, tools, and prompts are doing while systems are live. The central problem is that posture tooling can tell you what is deployed, but it cannot show you the actions an AI system takes once it starts interacting with data, APIs, and downstream workloads.

For identity teams, that creates a governance gap across NHI and emerging autonomous behaviour. The same access model that works for static service identities does not automatically explain who or what is acting, what data is touched, or how far an AI-driven action can propagate once execution begins.

Key questions

Q: How should security teams govern AI systems that make runtime decisions?

A: Security teams should govern AI systems as dynamic non-human identities, not static applications. That means tracking who or what initiated the action, what tools were used, which data was accessed, and whether the resulting behaviour stayed inside policy. Runtime governance has to combine identity, workload, and data context or it will miss the point where AI risk becomes operational.

Q: Why do AI agents create problems for existing IAM controls?

A: AI agents create problems because traditional IAM assumes access can be reviewed after it is granted and before it changes materially. When an agent can decide what to call, which tool to use, and when to act, the useful control point moves into runtime. That breaks review models built around stable, predictable access patterns.

Q: What breaks when teams only monitor prompts and not tool actions?

A: Teams miss the most important part of the event chain. A prompt may look harmless, but the model can still invoke tools, reach sensitive data, or trigger downstream automation that changes the risk profile. Without tool-action monitoring, security teams see input intent but not actual execution, which makes incident reconstruction incomplete.

Q: What should organisations do when AI activity crosses into cloud workloads?

A: They should treat the AI layer and the cloud layer as one control problem. If an AI system can pivot into applications, storage, or APIs, then the relevant question is not just whether the model is safe, but whether the connected workload and identity paths are governed as a single chain. That is where blast radius becomes visible.

Technical breakdown

Why posture visibility stops short of runtime AI security

Posture tools answer inventory questions: which models exist, where they run, and whether obvious misconfigurations are present. Runtime security answers a different question: what is the system doing right now, and what does that action mean for identity and data exposure? In AI environments, prompts, tool calls, and outputs form a chain of behaviour that can change from one session to the next. That makes static scanning insufficient because the risk sits in execution, not just configuration. The operational unit is no longer the asset alone, but the interaction between model, identity, tool, and data path.

Practical implication: Security teams need monitoring that correlates AI activity with workload context, not just asset posture.

Prompt-level analysis and why it still is not enough

Prompt-level analysis helps identify risky inputs such as secrets, PII, jailbreak attempts, or instructions that steer a model into unsafe behaviour. But prompts are only the first half of the transaction. A model can accept a benign prompt and still use legitimate tooling in a way that changes access patterns, data exposure, or downstream execution. That is why runtime AI security has to inspect both the input and the resulting action chain. The real control question is not only what was asked, but what the system chose to do with that request.

Practical implication: Teams should pair prompt inspection with action tracing across tools, APIs, and shared data paths.

MCP server interactions create a new identity and access boundary

Model Context Protocol servers connect AI systems to tools and data sources, which means they sit inside the decision path for AI-driven action. If security teams only watch prompts, they miss how the server invoked tools, what data it reached, and how output was consumed. That makes MCP a governance boundary as much as a technical one. The identity problem is that tool access is now mediated by an AI workflow that may be dynamic, chained, and opaque to traditional IAM review cycles. Without runtime correlation, the control plane sees usage, but not intent or consequence.

Practical implication: Map MCP access to the originating workload and process so tool use can be governed as part of identity risk.

Breaches seen in the wild

• Moltbook AI agent keys breach — Moltbook breach exposed 1.5M AI agent keys.
• AI LLM hijack breach — attackers used stolen AWS access keys to hijack Anthropic LLM models on Bedrock.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Runtime AI security is now an identity problem, not only an AI problem. Once an LLM-powered agent can call APIs, access records, or invoke tools, the security question shifts from static model posture to live identity behaviour. That makes AI activity part of the non-human identity estate, because the system is executing with access that must be governed like any other workload or service identity. Practitioners should treat runtime visibility as identity control, not an optional add-on.

Prompt controls do not solve tool-abuse risk because the consequential action happens after the prompt. A malicious or simply risky prompt can be visible, but the real exposure often comes from what the model does next with legitimate credentials and connected tools. This is why action tracing matters more than prompt review alone. Practitioners need to distinguish input risk from execution risk, because the latter is what turns an AI interaction into a security event.

Unified cloud and AI correlation is the real control gap the article exposes. AI systems do not stay inside an isolated AI layer. They pivot into cloud workloads, data stores, and downstream services, so a security team that separates AI monitoring from cloud monitoring will miss the attack path. The governance implication is that identity, workload, and data controls have to be evaluated together, not as independent silos.

AI runtime visibility should be measured by whether teams can explain who acted, what was accessed, and what tool chain executed. The article’s core problem is not a lack of dashboards. It is the absence of a reliable line from AI activity to accountable identity and business impact. Practitioners should view incomplete attribution as a governance failure, because if the actor and action chain cannot be reconstructed, neither access review nor incident response can be trusted.

MCP server activity is becoming a named concept for security teams: the runtime delegation gap. That is the space between a model receiving a request and the tool-mediated actions that follow. It matters because the risk is created by delegation in motion, not by configuration alone. For practitioners, that means governance must extend into the runtime chain where tool selection, data access, and output consumption actually happen.

From our research:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
• Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
• As AI runtimes expand, Top 10 NHI Issues helps teams connect runtime behaviour back to governance gaps in the broader non-human identity estate.

What this signals

Runtime AI governance will increasingly be judged by evidence quality, not dashboard count. If teams cannot reconstruct which identity initiated AI-driven activity, what tools were invoked, and which data paths were touched, then their control environment is still incomplete. The programme signal to watch is whether AI telemetry can support access review, investigation, and accountability across the same workflow.

The runtime delegation gap will become a standard failure mode in AI security programmes. AI systems that can act through tools create a new boundary where prompt review alone is insufficient. The governance test is whether organisations can follow a request from input to action and back to ownership without losing context.

With 53% of MCP servers exposing credentials through hard-coded values in configuration files, per the State of MCP Server Security 2025, the next control conversation is about runtime exposure plus secret handling, not one or the other.

For practitioners

• Correlate AI activity with workload identity Map prompts, tool calls, and outputs back to the originating workload, process, and human or agent identity so you can reconstruct the full action chain during review and incident response.
• Separate prompt risk from execution risk Treat sensitive prompts, jailbreak attempts, and secrets leakage as input signals, then validate what the model did after receipt by reviewing API calls, data access, and tool invocation logs.
• Include MCP servers in identity governance scope Inventory which MCP servers, tools, and skills can reach production data or applications, then apply access review and change control to those connections as you would for other privileged NHI pathways.
• Use runtime findings to refine AI policy boundaries Translate observed AI behaviour into policy scopes that reflect actual business usage, especially where models touch customer records, secrets, or sensitive operational datasets.

Key takeaways

• AI runtime security exposes a gap that posture tooling cannot close, because the most important risk appears when models start acting with real access.
• Evidence from the market shows the problem is already operational, with AI agents crossing intended scope in the majority of deployments.
• Practitioners should govern prompts, tools, and workload identity as one chain if they want AI risk to remain explainable and reviewable.

Key terms

• Runtime AI security: Runtime AI security is the discipline of monitoring and governing what AI systems do while they are live and interacting with real data. It focuses on prompts, tool calls, outputs, and downstream actions rather than only on deployment posture or configuration state.
• MCP server: An MCP server is a Model Context Protocol service that connects AI systems to tools and data sources. In practice, it becomes part of the identity and access path because it can mediate which resources an AI system can reach and what actions it can trigger.
• Prompt-level analysis: Prompt-level analysis examines AI inputs for risky content such as secrets, sensitive data, or malicious instructions. It is useful, but incomplete on its own, because the security impact often depends on what the model or agent does after receiving the prompt.
• Runtime delegation gap: The runtime delegation gap is the space between a request reaching an AI system and the tool-mediated action that follows. It is where governance often loses visibility, because the actor, the tool choice, and the resulting data access can all change during execution.

Deepen your knowledge

AI runtime security, prompt-level governance, and non-human identity boundaries are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building controls for agentic AI and cloud workloads, it is worth exploring.

This post draws on content published by Orca Security: The Runtime Gap: Why AI Security Can't Stop at Posture. Read the original.