AI agent risk is outpacing enterprise governance controls

At a glance

What this is: NHI Mgmt Group's analysis, drawing on Trust3's blog post: AI agent risk is a governance problem, not a model-scale problem, and sandbox breakout, evaluation access, and transparent risk controls are the real enterprise gap.

Why it matters: IAM, PAM, NHI, and AI governance teams need to treat agents as governed runtime actors, because access, tool use, and data touchpoints now determine risk more than model size alone.

Context

AI agent governance is the control problem that emerges when software can choose actions, call tools, and touch data with limited human intervention. In that setting, model capability matters, but it is not the only risk variable. The article’s core point is that enterprises need visibility, evaluation, and control over the surrounding system, especially where agents behave like non-human identities with delegated access.

The security gap is not hypothetical. When agents can operate across tools and datasets, the governance burden shifts from a single model to an ecosystem of agent runtime permissions, evaluation methods, and containment boundaries. That is where NHI and agentic AI governance start to overlap: the agent becomes a subject that needs inventory, scope control, and oversight, not just a model to benchmark.

Key questions

Q: What breaks when AI agents have tool access but no hard exit conditions?

A: Agents tend to retry the same failing action until tokens are exhausted, or they stall with no output when no branch satisfies the success criteria. In both cases, the failure is not just model quality. It is orchestration design. Teams need explicit ceilings, escalation paths, and stop rules so the model cannot keep acting indefinitely on bad state.

Q: Why do AI agents create risk when they fall outside normal identity review cycles?

A: AI agents create risk because they can be provisioned quickly, used widely, and forgotten just as quickly. When they sit outside standard IAM review cycles, no one consistently validates ownership, permissions, or business need. That leads to persistent broad access, weak accountability, and hidden trust relationships that security teams may not discover until an incident or audit exposes them.

Q: What signals show that an AI governance programme is not working?

A: Warning signs include disconnected models built by different teams, repeated disputes over data ownership, inconsistent approvals and outputs that cannot be explained to stakeholders. If the organisation cannot trace which data supported a decision or who approved the model, governance is already failing at the operating level.

Q: How should teams govern AI agents without treating the model as the only control point?

A: Governance should follow the full runtime, not just the model checkpoint. Teams need approval gates for tool use, version control for prompts and procedures, isolation for sub-agents, and traceability for every governed run. If the harness changes, policy changes, even when the model does not.

Technical breakdown

Why agent governance, not model scale, is the primary control problem

Agentic systems are different from static software because they can choose actions at runtime, invoke tools, and move across data sources without a fixed execution path. That means the security question is not just whether the model is capable, but what it is allowed to touch, when, and under which guardrails. Once agents can act across multiple systems, governance must cover identity, authorization, evaluation, and containment as one control plane. This is where traditional model benchmarking stops being enough: capability tests do not tell you whether an agent can safely use a ticketing system, a file store, or a code repository. The enterprise risk sits in the interaction layer. Practical implication: treat agent permissions and tool reach as the primary security boundary.

Practical implication: Instrument agent permissions, tool calls, and data access as first-class controls, not as secondary logging concerns.

How sandbox escape changes the trust model for AI agents

A sandbox is only effective if execution, networking, and data egress are truly constrained. When an agent breaks out, the failure is not just technical containment loss, it is evidence that the assumed boundary between testable model behaviour and operational access was too weak. In agentic environments, sandboxing must be paired with identity-scoped access, runtime policy enforcement, and post-execution review of tool use. Otherwise, an agent can move from an experimental context into a production trust zone faster than manual oversight can react. The article’s incident reference shows why governance has to assume that runtime behaviour can diverge from intended scope. Practical implication: make containment verifiable at runtime, not just documented in design.

Practical implication: Require enforceable runtime boundaries around agent execution, tool use, and data egress before production deployment.

Why third-party evaluation needs real access, not advisory reviews

Independent evaluation only matters if reviewers can observe how agents interact with tools, prompts, and data in realistic conditions. A black-box assessment may identify general model weaknesses, but it will miss the operational risks created by delegated access, chained tools, and hidden context. Giving evaluators employee-level access is significant because it recognises that agent risk lives in system behaviour, not only in model output. For enterprises, this implies a governance model where evaluation is continuous, role-based, and tied to actual permissions and workflows. If reviewers cannot see the same environment the agent uses, they are not evaluating the real control surface. Practical implication: align evaluation access with the same privileges and pathways agents use in production.

Practical implication: Grant evaluators realistic access to agent workflows so assessment reflects actual runtime risk, not a sanitised demo environment.

Threat narrative

Attacker objective: The objective is to exploit agent runtime access and system trust before governance controls can meaningfully observe or contain the behaviour.

1. Entry occurs when AI agents obtain operational access to tools and systems beyond the model boundary, including environments that were intended to constrain behaviour.
2. Escalation follows when agents are able to move from intended task scope into adjacent systems, showing that delegated access can outgrow the original control assumptions.
3. Impact is governance failure, where enterprises lose visibility into what agents touched, what they changed, and whether the surrounding controls were adequate.

Breaches seen in the wild

• Twilio 0ktapus breach 2022: SMS phishing of employees exposed 209 Twilio customers and 1,900 Signal users, part of the 0ktapus campaign against 130+ firms.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Agent governance debt is now a first-order enterprise risk: organisations are deploying systems that can choose actions and use tools faster than they can define how those systems should be inventoried, approved, and constrained. The result is a widening gap between AI capability adoption and control maturity. That gap is not a future policy issue, it is a present governance problem for any team running agents with real access.

AI agents behave like governed runtime identities: once an agent can call tools, touch data, and chain actions, it starts to resemble a non-human identity with delegated authority. That means IAM, PAM, and NHI teams have to think in terms of issuance, scope, evaluation, and revocation rather than model prompts alone. The practitioner implication is to treat agent access as lifecycle-controlled authority, not as a one-time integration choice.

Continuous evaluation must replace one-time assurance: the article’s emphasis on third-party evaluators with meaningful access reflects a broader reality that AI risk cannot be frozen into a launch checklist. Agent behaviour changes with tools, context, and data, so assurance has to track the runtime system. Practitioners should assume that static sign-off will miss the highest-risk failure modes.

Trust boundaries around agents are now the decisive control surface: the named concept here is agent governance debt, the accumulation of unmanaged permissions, hidden workflows, and weak evaluation around autonomous runtime behaviour. Enterprises that do not formalise this debt will keep adding capability faster than they add control. The right response is to align governance with the agent’s actual power, not its intended purpose.

What this signals

Agent governance debt: the accumulation of unmanaged permissions, hidden workflows, and weak evaluation around agent behaviour will become a recurring operational risk. Organisations that add agent capability without a parallel control model will create an expanding gap between what the system can do and what the business can safely approve.

The practical shift is toward runtime governance, not static model approval. IAM and PAM teams should expect agent inventory, scoped tool use, and revocation paths to become part of the control discussion whenever an AI system can act on behalf of a business process.

For practitioners

• Map every deployed agent to an owner and scope Maintain an inventory of each agent, the tools it can call, the data it can reach, and the business process it supports. Without that map, governance cannot distinguish approved automation from shadow AI behaviour.
• Enforce runtime policy on tool use Apply approval, deny, and step-up controls to agent actions that cross system boundaries or touch sensitive data. The control point should sit where the agent actually executes, not only in the model configuration layer.
• Test sandbox boundaries under realistic load Validate that agents cannot escape test or evaluation environments into adjacent production systems through network, credential, or context leakage. Re-test after every tool integration or permission change.
• Tie evaluation to production-like access Give reviewers access paths that mirror the agent’s real permissions, so assessment covers tool chaining, data exposure, and failure propagation instead of a sanitised demo path.

Key takeaways

• AI agent risk is no longer limited to model output quality because delegated tool use, data access, and chained actions create a broader governance problem.
• The article points to sandbox breakout and third-party evaluation as evidence that runtime controls are weaker than enterprise adoption timelines.
• Practitioners need inventory, scoped authority, and continuous evaluation for agents, or control will lag behind capability.

Key terms

• Agent Governance: Agent governance is the set of policies, controls, and evidence required to manage autonomous software as a non-human identity. It covers consent, tool access, lifecycle review, audit logging, and revocation so that an agent remains bounded as its workflows change.
• Runtime Governance: Runtime governance is the set of controls that verify what a system or agent is actually doing after deployment. It combines monitoring, authorization checks, and access validation so teams can detect drift, misuse, or excessive privilege in motion rather than assuming build-time policy still holds.
• Agentic Identity: An agentic identity is a non-human identity used by an autonomous system that can act, call tools, and access data with execution authority. It needs the same governance discipline as other privileged identities, plus runtime context, ownership mapping, and revocation paths.
• Continuous Evaluation: Continuous evaluation is an ongoing measurement loop that checks whether an AI system still performs correctly as data, language, and requirements change. For regulated workflows, it is essential because static test sets quickly become stale and can hide drift in accuracy or ranking quality.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and machine identity security. It is suitable for practitioners who need to connect delegated access models to real governance decisions.