AI agent governance is an unsolved identity problem

At a glance

What this is: FireCompass reports that senior security leaders see AI agent governance as an unsolved identity problem because existing controls do not fit runtime autonomy, repeatable errors, or unclear liability.

Why it matters: This matters because IAM, PAM, and security architecture teams now have to govern non-human actors that can make decisions and trigger actions faster than human review cycles can observe or stop them.

Context

AI agent governance is the problem of controlling non-human actors that can select actions, use tools, and execute work without waiting for a human to approve each step. In this article, FireCompass captures a closed-door roundtable view that current governance is not yet matched to how these systems behave in production.

The identity gap is the central issue. Human review cadences, stable intent assumptions, and ordinary output checks were designed for people, not for agents that can act at machine speed, repeat errors, and create shadow AI sprawl. That changes the governance problem from policy design to runtime control and evidence.

The article also broadens the issue beyond technical control. Liability is trending toward the deploying organisation, which means the governance model has to account for both operational containment and accountability evidence, even while the standards picture remains unsettled.

Key questions

Q: What breaks when AI agents are treated like standard human users?

A: You lose visibility into effective permissions, expected behaviour, and real blast radius. Human-centric controls can misclassify normal agent activity as compromise, or miss policy violations that happen entirely within legitimate access. The failure is not only technical, it is governance design that assumes a person is always behind the action.

Q: Why do AI agents create liability even when a vendor says the model acted autonomously?

A: Because the deploying organisation still selects the use case, grants the access, and operates the environment where the agent acts. The article’s examples show the direction of travel: courts and tribunals are treating outputs as organisational responsibility, not as the independent liability of the software. That makes attribution and evidence part of governance.

Q: What are the signs that an AI agent access model is becoming too permissive?

A: Warning signs include agents routinely reaching data outside their intended task, repeated approval exceptions, broad retrieval paths across unrelated systems, and outputs that expose information the user did not explicitly need. If the same agent is allowed to touch many data domains without clear policy boundaries, the control model is drifting away from least privilege.

Q: How should organisations compare human access controls with AI agent controls?

A: Human controls rely on judgment, training, and periodic review. AI agent controls need explicit action boundaries, continuous monitoring, and fast containment because the system can act without human interpretation in the loop. The difference is not just speed, but the loss of a stable human decision-maker inside the process.

Technical breakdown

Why human control loops fail for AI agents

Human governance assumes a decision maker can pause, reflect, and be reviewed before the next action. AI agents compress that loop: they can take a task, choose tools, produce outputs, and repeat the same error many times before a person notices. That turns a manageable mistake rate into an operational incident because the harmful unit is no longer one bad decision but a fast chain of decisions. The technical issue is not only reasoning quality, but the absence of a stable human-paced checkpoint between intent and execution.

Practical implication: move the control point earlier, to authorisation and tool access, instead of relying on post-action review.

What runtime identity means for AI agent governance

Runtime identity means the agent must be governed as a distinct identity at the moment it acts, not just at provisioning time. FireCompass’s roundtable aligned on per-agent identity, tightly scoped least privilege, and deterministic logs because the agent’s permissions and actions have to be attributable in real time. This is where identity becomes operational rather than administrative: if the system cannot prove which agent held which access when, accountability collapses. For AI agents, identity is not an onboarding formality. It is the mechanism that makes every action governable.

Practical implication: issue per-agent credentials, bind them to narrow scopes, and log every action with deterministic identity context.

Why shadow AI and hallucination become governance failures

Shadow AI emerges when approved agents can spawn additional agents or when teams lose inventory of what exists. Hallucination becomes a governance failure when the agent’s output cannot be cheaply verified, because the organisation ends up trusting fabricated findings, instructions, or decisions. The article’s practical consensus was that agent governance needs inventory, use-case approval, and controls that make destructive actions impossible unless explicitly allowed. The deeper pattern is that governance has to cover not just what the agent is permitted to do, but what it can create, claim, or trigger downstream.

Practical implication: maintain a live inventory of agents and constrain both delegation and self-propagation paths.

Breaches seen in the wild

• CoPhish OAuth phishing via Copilot Studio: Datadog showed Copilot Studio agents on a Microsoft domain can front OAuth consent phishing and forward stolen tokens; no victims reported.
• AI agent retail card theft campaign 2026: AI agents breached 27+ retailers for about $25 each, used cloud keys and a Secrets Manager dump, and stole 600,000+ payment cards.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI agent governance is an identity problem before it is an AI policy problem: The control failure is not that organisations lack opinions about AI, but that they do not yet govern agents as runtime identities with bounded authority. Human-style review, stable intent assumptions, and output checking were built for actors that wait to be inspected. The implication is that identity governance must shift to the moment of execution, not the moment of provisioning.

Human in the loop is a supervision model, not a scalable control model: The roundtable correctly treated human review as useful but operationally limited. Once agents begin acting at machine speed, the reviewer becomes the choke point and the exception handler, not the primary control. Practitioners need to understand that this is a structural limit, not a tuning problem.

Accountability is already moving toward the deploying organisation: The Munich and Air Canada examples in the article point to a simple governance reality: the organisation is likely to own agent outcomes even when the agent produced the error. That changes AI governance from a technology discussion into a liability evidence discussion. The practical implication is that auditability and attribution are now core governance requirements, not optional extras.

Shadow AI is the named concept security teams should track: If agents can spin up more agents, governance no longer ends at the first approved system. The article shows that one approval gap can become many unmanaged identities very quickly. Security teams should treat agent proliferation as an inventory and authority problem, because unmanaged growth is the path from experimentation to exposure.

Access review assumptions collapse when the actor can act before review: Access review processes were designed for access that persists long enough to be observed, challenged, and recertified. That assumption fails when an agent acquires access, completes work, and releases or reuses it within a short runtime window. The implication is that certification alone cannot be the primary safeguard for AI agent authority.

From our research library:

• 70% of organisations grant AI systems more access than they would give a human employee performing the exact same job, according to the 2026 Infrastructure Identity Survey.
• Read next: Zero Trust for AI Agents

What this signals

Access review assumptions collapse quickly in agentic environments: the governance model assumes a privilege survives long enough to be observed and recertified, but an agent can acquire, use, and discard authority inside a single task. That is why runtime enforcement matters more than periodic certification when the actor is non-human.

The practical question for IAM and PAM teams is no longer whether AI agents should be treated as users, but where their lifecycle differs enough to require a separate control path. Inventory, approval, scope, logging, and revocation all need to work at machine speed, or the organisation will end up governing the aftermath instead of the action.

For practitioners

• Treat each AI agent as a governed identity Assign a unique identity to every agent, scope access to a single approved purpose, and require an owner who can answer for its behaviour.
• Move controls to runtime authorisation Enforce permissions at the moment the agent attempts a tool call or destructive action, rather than assuming a pre-approved workflow stays safe.
• Build a live agent inventory Track which agents exist, who approved them, what they can access, and whether they can create additional agents or new execution paths.
• Make critical actions human-approved or impossible Reserve irreversible operations for explicit approval and remove the underlying tool capability when a task does not genuinely require it.
• Instrument deterministic logs and evidence Capture identity, tool use, input, output, and decision context in a form that supports audit, incident review, and liability reconstruction.

Key takeaways

• AI agent governance fails when organisations rely on human review cycles to control actors that can decide and act at machine speed.
• The article shows that accountability is trending toward the deploying organisation, which makes evidence, attribution, and runtime identity central to governance.
• Security teams need per-agent identity, tight privilege, and deterministic controls before destructive actions happen, not after they are discovered.

Key terms

• AI Agent Governance: AI Agent Governance is the set of policies, controls, and oversight practices used to direct how autonomous software agents behave. It defines allowed actions, approval paths, identity boundaries, logging, monitoring, and accountability so agent decisions remain traceable, constrained, and aligned with business, security, legal, and ethical requirements.
• Runtime Identity: Runtime identity is the practice of making identity and authorization decisions at the moment an action occurs. For agents and workloads, it means access is validated against live context, not only against the identity state set during onboarding or provisioning. That makes accountability and scope enforcement possible inside fast-moving workflows.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Assumption collapse: Assumption collapse occurs when a security model relies on a premise that no longer matches the actor's behaviour. In identity work, that usually means the model assumes a human-paced, stable access pattern, while the real actor can act faster, delegate differently, or change scope at runtime.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.