AI security governance fails when agents outpace control planes

At a glance

What this is: This is an analysis of AI security governance for agentic and LLM applications, showing that non-deterministic agent behaviour and MCP-connected tool chains require discovery, accountability and runtime enforcement.

Why it matters: It matters because IAM, PAM and security teams must treat AI agents as governed systems with scoped access, auditability and revocation, not as ordinary applications with static permissions.

By the numbers:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%).
• 92% agree governing AI agents is critical to enterprise security, yet only 44% have implemented any policies to do so.
• 96% of technology professionals identify AI agents as a growing security threat, and 66% believe this risk is immediate.

👉 Read Akto's analysis of AI security governance for agentic and LLM applications

Context

AI security governance is the control layer that sits between policy and execution when systems can make decisions, call tools and act without a human approving every step. The problem is not just model accuracy. It is the combination of non-deterministic output, delegated access and chained actions, which can create security outcomes that traditional application security does not see in time.

That gap matters for identity programmes because AI agents, service credentials and tool connections behave like non-human identities with changing runtime authority. Once an agent can reach MCP servers, databases or messaging tools, IAM and PAM teams need traceability, revocation and least-privilege boundaries that survive autonomous behaviour.

For enterprises that are already experimenting with agentic AI, this starting position is increasingly typical rather than exceptional. The governance challenge is no longer whether AI will be used, but whether the organisation can inventory, authorise and audit it before the blast radius grows.

Key questions

Q: What breaks when AI agents chain access across tools and services?

A: The original approval no longer describes the effective access path. Each individual hop may look legitimate, but the combined chain can extend privilege beyond what the organisation intended. The failure is visibility across the delegation chain, not just at the first credential issuance point.

Q: Why do AI agents create governance problems that normal access reviews miss?

A: AI agents can read, copy, transform, and re-share data after the original access decision, so a static review of entitlements does not capture downstream impact. Governance has to measure what the agent actually did with the data, not only whether the agent was allowed to see it. That is why lineage and activity evidence matter.

Q: How do security teams know if AI governance is working?

A: Look for evidence that access decisions are reviewable, permissions are revocable, and exceptions are not becoming permanent. If the team cannot explain who owns an AI workflow, what it can reach, and when its access was last reviewed, governance is incomplete. Control maturity shows up in traceability, not adoption volume.

Q: Which controls matter most when governing autonomous AI systems?

A: The most important controls are discovery, least privilege, runtime guardrails, audit logging and a clear decision owner for access changes. Together they reduce the chance that an agent can act outside approved scope and they give security teams evidence when an incident occurs.

Technical breakdown

Why agentic AI breaks traditional application security assumptions

Traditional application security assumes stable inputs, predictable workflows and a human-defined sequence of actions. Agentic AI breaks that model because an LLM can select tools, chain tasks and change behaviour based on context, prompt content or internal state. When an agent has access to APIs, databases or messaging systems, the risk is no longer limited to an unsafe answer. The risk is that the system acts on that answer before a human can intervene. In practice, that means governance has to move from content review to runtime control.

Practical implication: scope agent access as if each tool call were a privileged transaction, not a harmless application event.

How NIST AI RMF and ISO 42001 structure AI governance

Frameworks matter because AI governance needs a common language for ownership, risk and evidence. NIST AI RMF provides the govern, map, measure and manage structure for organising AI risk work, while ISO 42001 frames AI governance as a certifiable management system that can be sustained over time. Neither framework is a control list on its own. Their value is in forcing teams to define accountability, classify use cases, document risk decisions and show that controls are operating continuously rather than at deployment only.

Practical implication: map your AI programme to a governance framework before agent access expands, then use that mapping to drive evidence and review cycles.

Why discovery and inventory are the foundation for AI agent governance

You cannot govern what you cannot see. AI discovery tools aim to identify models, agents, MCP servers, embedded integrations and shadow AI use across cloud and endpoint environments. That inventory has to include the tools and data sources each agent can reach, because access scope is where governance becomes operational. Without that visibility, compliance teams cannot tell which systems are exposed, and security teams cannot distinguish sanctioned automation from unmanaged delegation.

Practical implication: maintain an inventory of AI agents, tools and data sources as a control asset, not as a documentation exercise.

Threat narrative

Attacker objective: The attacker wants to turn a trusted AI workflow into an execution path for data theft, unauthorized actions or broader access expansion.

1. Entry occurs when an attacker uses prompt injection, shadow integrations or compromised credentials to reach an agentic workflow or MCP-connected tool chain.
2. Escalation happens when the agent uses its delegated access to call additional tools, cross data boundaries or execute actions that were never intended in a single-step workflow.
3. Impact follows when the agent leaks data, triggers unauthorized business actions or exposes credentials that extend the attacker’s reach across connected systems.

NHI Mgmt Group analysis

AI security governance is becoming an identity problem as much as a model-risk problem. Once an agent can authenticate to tools, data stores or MCP servers, it behaves like a non-human identity with delegated authority. That means IAM, PAM and lifecycle controls become part of AI governance, not an adjacent concern. Organisations that treat agent access as application configuration will miss the real control point, which is identity, privilege and revocation.

Shadow AI is now a governance failure mode, not just an inventory gap. The article’s emphasis on discovery reflects a wider market shift: unmanaged agents and team-built integrations create authority without review, and authority without review is a security defect. This is where the concept of governance blind spots becomes useful. If security cannot enumerate the agent, its tools and its data paths, it cannot prove accountability or containment.

Runtime enforcement is the control plane that makes AI governance real. Policies and committees matter, but they do not stop a prompt injection event or a tool chain abuse in flight. The article correctly points toward guardrails, monitoring and posture management because the failure mode is active behaviour, not static code. For practitioners, the discipline shifts from approving AI to continuously constraining its authority.

AI governance maturity will increasingly be judged by evidence, not intent. NIST AI RMF and ISO 42001 both push organisations toward repeatable governance and auditable process. In practice, that means the question is whether a team can show agent inventory, approval history, tool scope and incident logs on demand. Mature AI governance is now a demonstrable control system, not a policy statement.

Agentic AI expands the blast radius of delegated access faster than most programmes can review it. The combination of autonomous action, chained tools and changing context means access reviews alone are too slow if they are not paired with runtime limits and telemetry. Security leaders should assume that AI governance is now a privilege management problem with model behaviour attached, not the other way around.

What this signals

The governance signal for practitioners is that AI controls now need to look more like identity controls than software release gates. If an agent can reach tools, data and external systems, the operating model has to include authorization scope, revocation, telemetry and incident reconstruction. The NIST AI Risk Management Framework provides a useful structure here, especially the govern, map, measure and manage loop, while the OWASP Agentic AI Top 10 helps teams translate risk into control choices.

Governance blind spots: the practical failure mode is not that organisations lack policies, but that they cannot prove which agents exist, what they can reach, and who approved that reach. That makes discovery and inventory the first operational control, not a supporting task. Teams that cannot enumerate MCP-connected agents will struggle to enforce least privilege or demonstrate compliance under pressure.

Agentic programmes will increasingly be judged on whether they can constrain runtime behaviour at the same speed as they onboard tools. That means posture management, guardrails and audit logging need to be integrated into the change process, not bolted on after deployment. The organisations that get ahead will be the ones that treat AI access as a lifecycle problem, not a one-time approval.

For practitioners

• Inventory every agent, model and MCP connection Create a live inventory that records each AI system, the tools it can call, and the data sources it can reach. Treat unsanctioned integrations and shadow AI as unmanaged privilege exposure, not as harmless experimentation.
• Assign explicit ownership for agent access decisions Put AI tool and data access decisions into a RACI model so one accountable owner approves each new delegation path. Do not leave access approval with the engineer who built the integration.
• Enforce runtime guardrails on tool use Limit which actions an agent can take, when it can take them and which outputs can leave the environment. Pair input filtering with output controls and per-tool allowlists so a compromised prompt cannot expand into uncontrolled execution.
• Log model decisions and tool calls for auditability Record prompts, tool invocations, data accesses and final actions in a format that supports incident reconstruction and regulatory review. Without those records, you cannot prove what the agent did or contained.
• Re-test agentic workflows after every material change Red-team prompt paths, tool chains and privilege boundaries whenever a model, prompt, connector or data source changes. Static review is not enough because agent behaviour and reachable assets change over time.

Key takeaways

• AI security governance fails when autonomous systems can acquire and exercise access faster than teams can review it.
• The strongest warning sign is scope drift, where AI agents act outside intended boundaries, access sensitive systems or reveal credentials.
• Discovery, least privilege, runtime guardrails and auditability are now the minimum controls for governable agentic AI.

Key terms

• AI security by design: AI security by design means building security, privacy, and access controls into AI systems from the start instead of adding them after deployment. In practice, it combines data governance, human oversight, documentation, and continuous monitoring so that model behaviour is auditable and bounded.
• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions — including calling APIs, writing code, and orchestrating other agents — with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.
• MCP Server: An MCP server is a tool endpoint that connects an AI agent to external systems and data sources through Model Context Protocol. Because it extends what the agent can reach, it becomes part of the identity and access surface and must be reviewed like any other privileged connector.
• Runtime Enforcement: Runtime enforcement is the practice of blocking malicious behaviour while software is running, rather than only detecting it after the fact. It monitors process activity, network actions, and privilege changes so a live attack can be interrupted at the point of execution.

What's in the full article

Akto's full post covers the operational detail this post intentionally leaves for the source:

• Framework-by-framework guidance for translating NIST AI RMF and ISO 42001 into an operating model
• Discovery workflow details for cataloguing MCP servers, AI agents and connected tools across cloud and endpoint environments
• Runtime guardrail examples for filtering inputs, limiting tool calls and constraining output behaviour
• Incident response considerations for prompt injection, data leakage and model behaviour drift

👉 Akto's full post covers discovery, accountability structures, runtime controls and auditability for AI governance.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security and secrets management. It is a practical fit for practitioners who need to govern delegated access across identity, automation and AI programmes.