Who is accountable when an LLM agent acts outside its intended scope?

Accountability sits with the organisation that grants the agent access, defines the workflow, and accepts the risk of delegated action. Regulators and auditors will expect evidence of ongoing testing, logging, and access control, especially where the agent can process personal data or influence regulated decisions.

Why This Matters for Security Teams

When an LLM agent acts outside its intended scope, the issue is not just that the model produced a bad answer. The real risk is that an authorised system used its delegated access in an unauthorised way, creating security, privacy, legal, and operational exposure in one event. That is why governance needs to cover the agent’s permissions, tools, prompts, logs, and human oversight, not just the model itself. The NIST AI Risk Management Framework is useful here because it treats AI risk as an organisational responsibility, not a model-only problem.

Many teams underestimate how quickly scope drift becomes a control failure. An agent that can query internal systems, generate transactions, or route decisions can move from productivity aid to delegated actor with real impact. If the intended scope is vague, then accountability is usually vague too, which weakens incident response and post-incident review. The organisation that deployed the agent is expected to define guardrails, test them, and prove they were enforced in practice.

In practice, many security teams encounter agent overreach only after a workflow has already executed outside policy, rather than through intentional control testing.

How It Works in Practice

Accountability for agent scope starts with design-time ownership and continues through runtime supervision. The business owner of the workflow typically owns the risk decision, while security, legal, privacy, and platform teams each control a different part of the system. That separation matters because an agent can inherit authority from its integration points, not from a single prompt. If it can access a ticketing system, payment tool, CRM, or code repository, then its effective authority is defined by those permissions.

Practically, organisations should document three things: what the agent is allowed to do, what it must never do, and what should happen when it is uncertain. This should be backed by logging, approvals, tool allowlisting, and periodic evaluation against expected behaviour. The OWASP Top 10 for Agentic Applications 2026 is relevant because it highlights failure modes such as excessive agency, tool misuse, and weak human oversight. For threat modelling, the MITRE ATLAS adversarial AI threat matrix helps teams think about manipulation, evasion, and misuse patterns that can push an agent beyond its intended bounds.

• Set explicit workflow boundaries and map them to approved tools, data sets, and decision types.
• Require step-up approval for high-impact actions, especially where money, access, or personal data is involved.
• Log prompts, tool calls, outputs, and human overrides so that decisions are reconstructable after the fact.
• Test the agent with adversarial and edge-case scenarios, including prompt injection and ambiguous instructions.

Where an agent uses identities, secrets, or tokens to operate, the scope problem becomes an NHI governance problem as well, which is why the OWASP Non-Human Identity Top 10 matters alongside access control. These controls tend to break down when agents are connected to many downstream tools through a shared service account, because attribution and containment become difficult to prove.

Common Variations and Edge Cases

Tighter control often increases operational overhead, requiring organisations to balance speed and autonomy against review, logging, and approval costs. That tradeoff becomes sharper when the agent supports customer service, engineering, or security operations, where teams want automation but still need reliable accountability. Current guidance suggests that the more consequential the action, the less tolerance there should be for unsupervised autonomy, although there is no universal standard for this yet.

One common edge case is shared responsibility across vendors and internal teams. A vendor may provide the model, another team may host the agent, and the business unit may define the workflow. Even then, accountability does not disappear; it is distributed, but the organisation that accepted the deployment risk still needs evidence of control ownership. Another edge case appears when an agent behaves within its technical permissions but outside its business mandate. That is not a tooling problem alone. It is usually a governance gap where the intended use was never translated into enforceable policy.

Where regulated decisions, personal data, or security-sensitive actions are involved, organisations should also align with the NIST AI 600-1 Generative AI Profile and, where appropriate, the CSA MAESTRO agentic AI threat modeling framework. Those references are especially useful when deciding where human approval is mandatory and where automated action is acceptable. Best practice is evolving, but the accountability line is already clear: if the organisation granted the access and failed to constrain it, the organisation owns the outcome.

Related resources from NHI Mgmt Group

• Who is accountable when an AI agent acts outside its intended scope?
• How can organisations tell whether an AI agent is acting outside its intended scope?
• Who is accountable when an AI agent exceeds its intended scope?
• Who is accountable when an agent opens an outbound connection outside its intended boundary?