What breaks when agent governance stops at the LLM layer?

Why This Matters for Security Teams

When governance stops at the LLM layer, security teams get a false sense of control. A model can produce policy-compliant output while the surrounding agent still calls a tool, fetches records, or submits a workflow that changes systems of record. The real risk is not the text alone, but the execution path that follows it. That is why NHI Management Group treats agent identity and runtime action control as a combined problem, not two separate ones.

This gap shows up clearly in current research. In the AI Agents: The New Attack Surface report, 80% of organisations said their AI agents had already performed actions beyond intended scope, while only 52% could track and audit the data those agents accessed. That is not a prompt issue. It is an execution and observability issue. The same pattern appears in the OWASP NHI Top 10, where agent behavior is treated as a broader control problem than model output alone.

Security leaders often miss this because the LLM appears to “behave” while the agent quietly exercises authority elsewhere. In practice, many security teams encounter the failure only after an agent has already touched production data or triggered an irreversible workflow, rather than through intentional testing.

How It Works in Practice

Effective governance must follow the agent beyond the model boundary and down into the tools, secrets, APIs, and workflows it can invoke. The practical shift is from “Is the response safe?” to “Is this specific action allowed, for this agent, in this context, right now?” That aligns with current guidance from the NIST AI Risk Management Framework and the CSA MAESTRO agentic AI threat modeling framework, both of which push control ownership toward runtime risk, not just content moderation.

In practice, that means treating the agent as a workload identity with narrowly scoped, short-lived access. A strong pattern is:

• Issue just-in-time credentials per task, not long-lived secrets that persist across goals.
• Bind tool calls to workload identity and policy evaluation at request time, rather than assuming role membership is enough.
• Use real-time authorization for each action, so the agent can be approved for one operation and blocked from the next.
• Log the execution path, not just the prompt and response, so investigators can see what was reached, changed, or exfiltrated.

This is where agent governance diverges from classic chat safety. A harmless prompt can still lead to an unsafe API call, especially when the agent chains tools or follows an injected instruction hidden in external content. The issue is not limited to one platform; it applies whenever an agent has broad OAuth scopes, shared service accounts, or static API keys. The CoPhish OAuth Token Theft via Copilot Studio case is a useful reminder that token exposure and action authority often move together.

These controls tend to break down in environments where agents operate across many tools with inconsistent identity models, because one unchecked connector can bypass the policy layer entirely.

Common Variations and Edge Cases

Tighter runtime controls often increase integration overhead, requiring organisations to balance safety against developer velocity and operational complexity. There is also no universal standard for this yet, so teams need to distinguish settled practice from emerging guidance. For example, policy-as-code and per-request authorization are increasingly recommended, but the exact implementation varies by stack, risk appetite, and tool ecosystem.

One common edge case is read-only versus write-capable agents. Read-only access still matters, because sensitive data exposure can occur without any system change, but write paths demand stronger guardrails, explicit approval, and narrower TTLs. Another is delegated human-assisted agents, where an operator assumes the model layer is the only variable. In reality, the agent may still hold broad credentials from a prior step. That is why the NIST Cybersecurity Framework 2.0 remains useful as a baseline, but it must be applied to the full execution chain, not just the model endpoint.

Best practice is evolving around ephemeral secrets, workload identity, and continuous authorization for autonomous systems. Static RBAC alone is not enough when the agent’s goals change at runtime. The most reliable programs treat the LLM as one component in a broader control plane, then verify every downstream action against business context, data sensitivity, and current trust conditions. Where agents span many services and third-party integrations, governance usually degrades fastest at the connector boundary, because that is where identity, authorization, and observability are least consistent.

Related resources from NHI Mgmt Group

• What breaks when security teams rely only on DSPM for AI agent governance?
• What is the difference between human identity governance and AI agent governance?
• What is Microsoft Agent 365 in AI agent governance?
• What breaks when an agent identity layer does not include access governance?