Accountability sits with the organisation that granted the permissions, defined the workflow, and failed to impose approval or admission gates. Frameworks such as NIST CSF and NIST AI RMF both assume governance over behaviour, not just model output.
Why This Matters for Security Teams
Accountability for a malicious agent workload is rarely a model problem alone. It usually traces back to the organisation that allowed the agent to act with excessive authority, unclear approval paths, or broad tool access. That makes this a governance and control issue, not a debate about whether the model “intended” harm. The practical question is who approved the permissions, who owned the workflow, and who failed to contain the blast radius.
This is why guidance from NIST AI Risk Management Framework and the OWASP Agentic AI Top 10 focuses on governance, authorization, and monitoring rather than treating agents like passive software. NHIMG research on AI Agents: The New Attack Surface report found that 80% of organisations report AI agents have already acted beyond their intended scope, which is a strong signal that ownership gaps are already operational, not theoretical. In practice, many security teams encounter accountability only after an agent has already chained tools, touched sensitive systems, or triggered an incident.
How It Works in Practice
For autonomous workloads, accountability should be mapped to the control plane, not the model itself. The organisation is accountable when it defines the workflow, grants the permissions, and fails to impose runtime gates that constrain what the agent can do. That usually means security, platform, and application owners must jointly define allowed actions, approval points, logging requirements, and rollback conditions.
Current practice increasingly relies on workload identity and runtime policy rather than static human-style roles. Standards such as the SPIFFE workload identity specification help establish cryptographic proof of what the agent is, while policy engines evaluate whether a specific action should be allowed in that moment. Best practice is evolving toward just-in-time credentials, ephemeral secrets, and request-time authorization so the agent receives only the access needed for a single task. NHIMG’s Ultimate Guide to NHIs underscores why short-lived machine credentials matter: autonomous systems fail in ways that are fast, chained, and difficult to unwind once standing privileges are in place.
- Assign a named business and technical owner for every agent workflow.
- Use admission gates for deployments, tool calls, and outbound actions that can affect data or infrastructure.
- Issue short-lived credentials per task, then revoke them automatically on completion.
- Log tool use, prompt-to-action mappings, and policy decisions for audit and investigation.
- Review whether the agent can access production, secrets, or third-party APIs without human approval.
These controls tend to break down in highly dynamic environments where agents can discover new tools, trigger nested workflows, or operate across multiple tenants without a consistent policy layer.
Common Variations and Edge Cases
Tighter approval and admission control often increases operational overhead, so organisations must balance speed against containment. That tradeoff is especially visible in CI/CD automation, customer-facing support agents, and multi-agent orchestration where a single delayed approval can slow delivery.
There is no universal standard for this yet, but current guidance suggests the same accountability pattern holds across most cases: the entity that authorised the capability remains responsible for the harm caused by misuse, unless a contract or regulation assigns duties differently. If an agent was given production database access, token export rights, or a deploy path without human review, accountability usually sits with the organisation that created that exposure. The strongest evidence comes from incidents such as Replit AI Tool Database Deletion and Amazon Q AI Coding Agent Compromised, where the issue was not only malicious input but also insufficient containment of execution authority.
For legal and governance teams, the hard edge case is shared control. A vendor may supply the model, but the deploying organisation still owns the workflow, data, and permissions in most enterprise deployments. That is why operational accountability should be documented before release, not after an incident.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, CSA MAESTRO and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A1 | Addresses excessive tool access and unsafe agent actions. |
| CSA MAESTRO | M1 | Covers agent workflow governance and runtime control points. |
| NIST AI RMF | Sets governance expectations for AI behaviour and accountability. | |
| NIST CSF 2.0 | PR.AC-4 | Least privilege and access control are central to agent accountability. |
| OWASP Non-Human Identity Top 10 | NHI-03 | Ephemeral secrets and rotation reduce blast radius from rogue agent use. |
Assign governance, testing, and monitoring duties for agent actions before deployment.
Related resources from NHI Mgmt Group
- Why is single-provider AI agent governance not enough for enterprise security?
- Who is accountable when an AI agent installs a malicious skill?
- Who is accountable when an AI agent or build pipeline introduces malicious code?
- Who is accountable when an AI agent or automation system installs a malicious dependency?
