Subscribe to the Non-Human & AI Identity Journal

Who is accountable when an AI agent takes an unsafe action?

Accountability should sit with the business owner of the agent, the team that provisioned the access, and the control owners responsible for monitoring and revocation. If no one can answer who approved the identity, the scope, and the oversight model, the governance framework is not complete enough for production.

Why This Matters for Security Teams

An unsafe action by an AI agent is not just a model-quality issue. It is an identity, access, and governance event. Once an agent can chain tools, request data, or trigger workflows, accountability has to follow the business owner, the provisioning team, and the control owners who can detect and revoke access. That is why current guidance increasingly ties agent governance to OWASP Agentic AI Top 10 and the NIST AI Risk Management Framework, rather than treating agents like ordinary service accounts.

The practical risk is that an agent’s behaviour is goal-driven, not static. A role that looked safe at provisioning time can become unsafe after a prompt change, a tool update, or a new upstream data source. NHIMG research on OWASP NHI Top 10 shows why agentic systems need explicit identity, scope, and revocation controls, while the SailPoint report notes that 80% of organisations have already seen AI agents act beyond intended scope.

In practice, many security teams discover this ownership gap only after an agent has already accessed data or executed an unintended workflow, rather than through intentional governance design.

How It Works in Practice

For autonomous agents, accountability should be mapped to decision and control points, not just to the model itself. The business owner defines the task boundary and acceptable outcomes. The team that provisions access decides which systems, secrets, and tools the agent can reach. The control owner defines monitoring, alerting, and revocation. That split matters because an AI agent is an execution entity, not a passive application, so static RBAC alone is often too blunt for real-time behaviour.

Current best practice is evolving toward intent-based authorisation: the agent asks for access at runtime, and policy is evaluated against context such as task, risk, data sensitivity, and time window. That often means CSA MAESTRO agentic AI threat modeling framework style analysis, plus request-time policy decisions aligned to the OWASP Top 10 for Agentic Applications 2026. In operational terms, teams should use workload identity for the agent, short-lived JIT credentials for each task, and ephemeral secrets with tight TTLs so access expires automatically when the job ends.

  • Use workload identity to prove which agent instance is acting.
  • Issue JIT credentials only for the approved task and revoke them on completion.
  • Evaluate policy at request time, not only at enrollment or deployment.
  • Log the owner, approver, context, and tool chain for every privileged action.

NHIMG’s AI LLM hijack breach coverage and DeepSeek breach analysis both reinforce the same point: when secrets are exposed or permissions are overbroad, autonomous systems can amplify the blast radius far faster than a human workflow. These controls tend to break down in tool-rich environments with weak service-to-service visibility because agents can chain actions faster than humans can review them.

Common Variations and Edge Cases

Tighter controls often increase operational overhead, so organisations have to balance safety against delivery speed. That tradeoff becomes sharper when multiple teams share one agent, when the agent spans SaaS and cloud workloads, or when it is embedded in an engineering pipeline that expects high autonomy. In those cases, accountability can blur unless every privileged action has a named owner and a revocation path.

There is no universal standard for this yet, but guidance consistently points in the same direction: separate the person who approved the capability from the person who provisioned the privilege and the person who monitors the outcome. That is especially important where an agent can trigger outbound payments, modify infrastructure, or retrieve sensitive records. The NIST AI Risk Management Framework and NHIMG’s Analysis of Claude Code Security both support the idea that governance must be continuous, not one-time.

Where mature teams get this right, they treat the agent like a high-risk workload identity with narrow scope, per-task access, and monitored revocation. Where it fails, the common pattern is an “owner” listed in a ticket, but no one who can actually answer for the unsafe action in real time.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

OWASP Agentic AI Top 10 and CSA MAESTRO address the attack and risk surface, while NIST AI RMF set the governance and control requirements practitioners need to meet.

Framework Control / Reference Relevance
OWASP Agentic AI Top 10 A2 Directly addresses unsafe agent actions and overbroad autonomy.
CSA MAESTRO Maps accountability to agent threat modeling and runtime control ownership.
NIST AI RMF Covers governance for AI accountability, oversight, and risk management.

Define accountable owners, review loops, and escalation paths for each agent capability.