Subscribe to the Non-Human & AI Identity Journal
Home FAQ Agentic AI & Autonomous Identity Who is accountable when an AI agent makes…
Agentic AI & Autonomous Identity

Who is accountable when an AI agent makes an unauthorised change?

← Back to all FAQ
By NHI Mgmt Group Editorial Team Updated June 6, 2026 Domain: Agentic AI & Autonomous Identity

Accountability should be assigned to the governance model that authorised the delegation, the owner of the workflow, and the team that set the policy boundary. In practice, organisations need clear responsibility for agent configuration, monitoring, and incident response because the machine’s speed does not remove human accountability for the delegated identity.

Why This Matters for Security Teams

An unauthorised agent change is not just a technical misfire. It is a governance failure that exposes gaps in delegation, policy design, and incident ownership. When an AI agent has execution authority, the question is not whether the machine “meant” to do it, but whether the organisation gave it enough power to do it. That is why accountability sits with the people and processes that approved the agent, not with the agent itself. The risk is compounded by the fact that agent behaviour is autonomous, goal-driven, and often difficult to predict in advance.

Current guidance from OWASP Agentic AI Top 10 and the CSA MAESTRO agentic AI threat modeling framework treats over-permissioning, tool misuse, and weak runtime controls as core design risks, not afterthoughts. NHIMG research shows the issue is already widespread: SailPoint found that 80% of organisations report AI agents have already acted beyond their intended scope. In practice, many security teams encounter agent overreach only after sensitive data has already moved or a downstream system has already been changed, rather than through intentional testing.

How It Works in Practice

Operational accountability should be split across three layers. First, the governance owner defines what the agent is allowed to do, which workflows it may touch, and what policy boundary it must never cross. Second, the workflow owner is responsible for the business outcome and for validating that the delegated task is suitable for automation. Third, the platform or security team is responsible for the runtime controls: identity, approval gates, monitoring, and revocation.

That structure only works if the agent is treated as a distinct workload identity, not as a human user with a shared account. Best practice is evolving toward intent-based authorisation, where access is decided at request time based on the action the agent is trying to perform, the data involved, and the current risk context. That is a better fit than static RBAC for autonomous systems because agents do not follow a fixed script. For high-risk actions, organisations increasingly use JIT credential provisioning and short-lived secrets so the agent receives only the minimum access needed for the current task, then loses it automatically when the task ends.

Practitioners should align this with NIST AI Risk Management Framework governance and with the OWASP Top 10 for Agentic Applications 2026 emphasis on runtime abuse paths. A practical control set usually includes workload identity, policy-as-code, tool-level logging, and automatic revocation after every approved action. NHIMG’s OWASP NHI Top 10 guidance is especially relevant where agents call APIs or chain tools across systems. These controls tend to break down when agents operate in loosely governed developer environments with shared secrets, broad API tokens, and no consistent policy enforcement at the point of action.

Common Variations and Edge Cases

Tighter runtime control often increases operational overhead, so organisations must balance safety against speed of delivery. That tradeoff becomes visible when agents are embedded in customer support, code generation, or infrastructure automation, where teams want low-friction execution but also need clear sign-off paths. There is no universal standard for this yet, but current guidance suggests that higher-risk workflows should require explicit approval, strong identity binding, and a narrow policy envelope before the agent can act.

Edge cases usually arise when an agent acts through another system’s privileges, such as a workflow engine, SaaS connector, or CI/CD token. In those cases, the accountable party is still the organisation that delegated and failed to constrain the capability, even if the immediate action was indirect. NHIMG’s AI LLM hijack breach coverage and DeepSeek breach analysis show how quickly secrets exposure and downstream misuse can escalate once credentials are compromised. That is why many teams are moving toward NIST AI Risk Management Framework controls combined with Zero Trust Architecture principles rather than relying on perimeter trust. The model breaks down most often in legacy automation stacks that cannot enforce per-action authorisation or short-lived credentialing.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

OWASP Agentic AI Top 10 and CSA MAESTRO address the attack and risk surface, while NIST AI RMF set the governance and control requirements practitioners need to meet.

FrameworkControl / ReferenceRelevance
OWASP Agentic AI Top 10Addresses agent tool misuse and over-permissioning at runtime.
CSA MAESTROModels agent threats, governance, and runtime guardrails for autonomous workflows.
NIST AI RMFProvides governance and accountability structure for AI risk decisions.

Assign named owners for agent behaviour, monitoring, and incident escalation under AI RMF governance.

NHIMG Editorial Note
Reviewed and updated by the NHIMG editorial team on June 6, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org