Who is accountable when an unsanctioned AI agent causes an incident?

Why This Matters for Security Teams

An unsanctioned AI agent is not just “another app” gone wrong. It is an autonomous workload with tool access, latent privileges, and the ability to chain actions faster than a human operator can intervene. That changes accountability from a simple user-owned incident into a governance failure across business ownership, technical ownership, and control ownership. Current guidance from NIST AI Risk Management Framework and OWASP Agentic AI Top 10 points in the same direction: if no one can explain why the agent existed, what it could do, and who approved it, then the organisation has already lost control of the risk.

NHIMG research shows how quickly this becomes material. In AI Agents: The New Attack Surface report, 80% of organisations said their AI agents had already acted beyond intended scope, while only 44% had any policies governing them. That gap matters because incident accountability is impossible to assign cleanly after the fact if there was no approval chain, no monitoring owner, and no named system steward in the first place. In practice, many security teams discover this only after the agent has already accessed data or executed a tool call that nobody expected.

How It Works in Practice

For autonomous agents, accountability should be built around the control plane, not just the person who typed a prompt. The business owner is accountable for the decision to deploy the agent; the technical owner is accountable for its integration, identity, and runtime boundaries; and the control owners are accountable for approvals, logging, and alerting. That aligns with CSA MAESTRO agentic AI threat modeling framework, which treats agent behaviour as something that must be designed, instrumented, and continuously assessed.

In operational terms, the safer model is intent-based authorisation with just-in-time credentials. The agent should present workload identity, such as an OIDC-based service identity or SPIFFE/SPIRE-issued identity, then receive short-lived secrets only for the specific task it is allowed to complete. Static RBAC alone is not enough when behaviour is goal-driven and variable. The real decision should happen at request time, using policy-as-code and context: what the agent is trying to do, which system it wants to touch, and whether the action matches the approved intent. That is consistent with MITRE ATLAS adversarial AI threat matrix and the NIST AI Risk Management Framework, both of which emphasise tracing risk to behaviour, not assumption.

• Use named ownership for every agent, including the approving business unit and the platform team.
• Issue ephemeral credentials per task, not reusable secrets with broad lifetime.
• Log every tool call, data access, and privilege escalation attempt to a tamper-evident audit trail.
• Block unsanctioned agents at the identity and orchestration layer, not only at the endpoint or SIEM layer.

These controls tend to break down in highly distributed environments where agents can spawn other agents, reuse shared tokens, or call legacy systems that do not support fine-grained policy evaluation.

Common Variations and Edge Cases

Tighter control often increases deployment friction, requiring organisations to balance speed against assurance. That tradeoff is real, especially where product teams want fast experimentation but security teams need clear attribution. Current guidance suggests that accountability can be shared, but it should never be ambiguous: if an agent was approved through a sanctioned workflow, the approver and control owners remain accountable for the guardrails; if it was deployed outside governance, the business sponsor and technical operator who enabled the connection still own the remediation path.

Edge cases usually appear in semi-sanctioned environments. A vendor-managed agent, a developer sandbox, or an AI assistant embedded in a business workflow may look harmless until it inherits enterprise credentials or access to production data. That is why NHIMG emphasises non-human identity discipline in The 52 NHI breaches Report and OWASP NHI Top 10: once a machine identity can act, it must be governed like a privileged actor, not like a passive integration. Where there is no universal standard for agent accountability yet, best practice is to define it in advance through policy, contracts, and incident runbooks rather than debate it after an event.

In the hardest cases, the answer is not “who clicked approve” but “who owned the risk decision to let an autonomous system act on enterprise assets.” If that cannot be answered quickly, the control model has already failed.

Related resources from NHI Mgmt Group

• Who should be accountable when an AI agent causes a security incident?
• Who is accountable when an autonomous AI agent causes a security incident?
• Who is accountable when an AI agent exposes credentials or changes identity state?
• Who is accountable when an AI agent accesses the wrong data?