AI governance for agentic systems breaks at runtime, not policy

At a glance

What this is: This analysis argues that AI governance breaks when autonomous agents move from generating text to taking actions across connected systems.

Why it matters: It matters because IAM, PAM, and governance teams now need to control runtime authority, inventory, and escalation paths across non-human and agentic identities, not just outputs.

👉 Read Lasso Security's analysis of AI governance challenges in agentic AI

Context

AI governance is the set of controls, oversight, and policy boundaries used to manage what AI systems can access and do. In agentic environments, the core problem is no longer content moderation or prompt review. It is whether the identity behind the agent can act across systems faster than governance can observe or constrain it.

The article describes a shift from chat-oriented oversight to consequence-oriented control. That shift matters for non-human identity, because every connected model, API, and workflow becomes part of the agent's effective access boundary. Teams that already struggle with shadow AI and unmanaged machine access will recognise the same inventory problem in a more dynamic form. See the [Top 10 NHI Issues](https://nhimg.org/top-10-non-human-identity-issues) for the recurring control failures that show up when identity expands faster than governance.

Key questions

Q: How should security teams govern AI agents that can act across multiple systems?

A: Security teams should govern AI agents as non-human identities with explicit action boundaries, not as chat interfaces. That means discovering each agent, mapping its inherited permissions, restricting tool reach, and monitoring runtime actions across every connected system. If an agent can query, update, and trigger workflows, governance has to follow those actions end to end.

Q: Why do agentic AI systems complicate least privilege?

A: They complicate least privilege because the agent's effective scope is not fully knowable at provisioning time. Tool selection, state, and model behaviour can change what the agent does after access is granted. As a result, static entitlements may appear reasonable on day one but become too broad or too narrow once the agent is live.

Q: What breaks when AI governance only monitors prompts and outputs?

A: Prompt and output monitoring misses the moment where the real risk occurs, which is execution. An agent can produce a harmless-looking response while still calling APIs, updating records, or chaining actions across systems. Governance that stops at the conversation layer cannot see the blast radius created by runtime action.

Q: Who is accountable when an autonomous agent causes business harm?

A: Accountability sits with the organisation that granted the agent its access, defined its guardrails, and failed to monitor its runtime behaviour. In practice, responsibility spans the AI owner, the identity team, and the control owners for every connected system the agent can touch. Governance must make that chain explicit before incidents occur.

Technical breakdown

Why agentic AI governance is different from output monitoring

Traditional AI governance assumes the control point is the response. Agentic AI moves the control point into execution, where an agent can decide what tool to call, what system to query, and what downstream action to trigger. That means the relevant security boundary is no longer the prompt or output alone. It is the full action sequence, including the systems touched, the permissions inherited, and the timing of the action. Once an agent can update records or trigger workflows, governance has to account for business impact, not just model correctness.

Practical implication: define policy around actions, permissions, and downstream system reach, not just around text output.

Agent inventory and access drift in connected systems

Agentic deployments often spread across code repositories, cloud platforms, and low-code environments at the same time. That creates a live inventory problem because agents can change models, tools, and API connections without a visible code change. In identity terms, the effective subject is a moving target. Governance depends on knowing which agents exist, what they can reach, and which inherited permissions have accumulated over time. Without continuous inventory, policy becomes aspirational because it cannot be mapped to actual runtime access.

Practical implication: maintain continuous discovery for agents, their tools, and their reachable systems before trying to enforce least privilege.

Fragile intent and multi-turn manipulation

Agent controls fail differently from classic application controls because the failure is often progressive rather than abrupt. A single malicious instruction may not work, but a sustained exchange can gradually shift the agent's intent until it acts outside scope. That makes multi-turn testing essential. The issue is not simply malicious input. It is that the model's decision boundary can be worn down during runtime, especially when the agent has real authority and broad tool access. Governance has to test for intent fragility, not just known bad prompts.

Practical implication: include multi-turn adversarial testing and runtime detection for scope drift, not only one-shot prompt filters.

Threat narrative

Attacker objective: The objective is to use an autonomous agent's legitimate access and execution speed to cause cross-system business impact before human oversight can intervene.

1. Entry occurs when a legitimate agent connects to databases, external APIs, workflows, and other enterprise systems with broad inherited permissions.
2. Escalation happens when the agent chains tool calls across those systems and executes actions that human reviewers cannot interrupt at the same speed.
3. Impact follows when a compromised or misdirected agent updates records, moves funds, or triggers business processes across multiple platforms before the organisation can contain the blast radius.

Breaches seen in the wild

• Moltbook AI agent keys breach — Moltbook breach exposed 1.5M AI agent keys.
• AI LLM hijack breach — attackers used stolen AWS access keys to hijack Anthropic LLM models on Bedrock.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Agentic AI creates an identity governance problem, not just an AI policy problem. Once an agent can act across systems, the question is no longer whether its output is acceptable. The question is whether its runtime authority is bounded well enough for IAM, PAM, and governance controls to make sense. That moves the issue into identity security, where inventory, entitlement scope, and escalation boundaries are the real control plane. Practitioners should treat agent access as governed identity, not as a content moderation problem.

Least privilege is increasingly hard to define at provisioning time for agentic systems. Agentic behaviour is non-deterministic because tool selection, execution order, and downstream effects depend on state, history, and model behaviour. That means the intended scope can drift after deployment, even if the original permission set looked reasonable. The implication is that static entitlement assumptions are losing validity for agentic actors, and governance has to account for changing runtime context.

Runtime governance gap: The control model was designed for systems that could be reviewed after action, but autonomous agents can complete the full sequence before review begins. Human-paced approval cycles were built for actors that remain stable long enough to be observed, certified, and remediated. When the agent decides and acts in one uninterrupted flow, the governance window collapses. Practitioners must rethink any control that depends on a persistent, reviewable state.

Agent-to-agent delegation compounds blind spots across the identity chain. When one agent instructs another, conventional monitoring loses the stable human operator it expects to track. That makes accountability and blast-radius analysis harder because the chain of action is distributed across non-human actors. The field needs to stop treating delegation as a workflow convenience and start treating it as an identity boundary problem with compounded risk. Practitioners should map those chains before they become operational dependencies.

The market signal is clear: governance is moving from policy documents to continuous verification. The article's own framework for discovery, assessment, testing, and runtime enforcement reflects where the discipline is heading. That direction aligns with [OWASP Agentic AI Top 10](https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/) and the [NIST AI Risk Management Framework](https://www.nist.gov/artificial-intelligence/ai-risk-management-framework), because both frameworks treat AI risk as something to manage through ongoing governance rather than static approval. Practitioners should expect agent identity controls to become part of mainstream security architecture.

From our research:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
• 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
• That combination of scope drift and audit blindness shows why agent governance has to move from policy intent to runtime identity control, as explored in Analysis of Claude Code Security.

What this signals

Runtime governance gap: agentic programmes fail when access review cadences assume privileges remain stable long enough to be certified. The more the agent can change tools, context, and execution path mid-session, the less useful periodic review becomes as a control signal.

The practical response is to align AI oversight with identity discipline already used for other non-human actors: continuous discovery, bounded access, and evidence-based enforcement. That is why NHI and agentic AI governance are converging into the same operational problem space, even if the tooling remains fragmented today.

For practitioners

• Build a live agent inventory Track every agent across code repositories, cloud platforms, and third-party services, including the models they run, the tools they can call, and the APIs they inherit. A point-in-time register is not enough when agents change without a code commit.
• Separate output review from action control Define control points for database queries, workflow triggers, record updates, and external API calls. The agent's text output may look safe while its runtime actions create the real exposure.
• Test for fragile intent under sustained prompting Use multi-turn adversarial testing to see where an agent's intent breaks down under persistence, state changes, and tool pressure. Single-turn prompt filters miss the failure mode that matters most in agentic environments.
• Map agent-to-agent delegation chains Document where one agent passes instructions to another, then identify which downstream actions cross trust boundaries or escalate privileges. That mapping is the only way to understand compounded blast radius.
• Enforce adaptive runtime guardrails Tie guardrails to observed behaviour and update them as tools, models, and permissions change. If the policy cannot change as fast as the agent, it will lag behind the actual risk.

Key takeaways

• Agentic AI turns governance into an identity problem because the agent's actions, not just its outputs, create the security exposure.
• Scope drift, audit gaps, and inherited permissions are the main reasons static AI policies fail once agents begin taking real actions across systems.
• Practitioners should move to continuous discovery, runtime guardrails, and multi-turn testing if they want governance to keep pace with autonomous behaviour.

Key terms

• Agentic AI: AI systems that can decide and execute actions across tools and systems, not just generate text. In governance terms, the important issue is not model sophistication but runtime authority, because action-taking introduces identity, access, and accountability concerns that prompt-only controls cannot cover.
• Runtime governance: The control layer that evaluates and constrains what an AI system does while it is operating. For agentic systems, runtime governance matters because static approval at deployment cannot predict every tool call, state change, or downstream action the agent may take later.
• Fragile intent: The point at which an agent can be manipulated into acting outside its intended scope through persistence, context shifts, or pressure. This is an operational concept, not a model-quality score, and it is especially relevant when the agent has access to real enterprise systems.
• Agent inventory: A live record of every AI agent in the environment, including what it runs, what it can access, and which systems it can affect. The term matters because unmanaged or undiscovered agents create shadow AI risk and make governance controls impossible to enforce reliably.

Deepen your knowledge

AI governance for agentic systems is a core topic in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building controls for agents that act across connected systems, this course is a practical place to start.

This post draws on content published by Lasso Security: AI Governance Challenges in the Age of Agentic AI. Read the original.