Human-in-the-Loop for AI Agents: Approval Gates

Human approval is the control most often promised for AI agents and the one most often implemented badly: as a prompt the agent can ignore, a dialog people click through, or an approval the agent can forge. Done well, a human-in-the-loop gate blocks a high-impact action until an authorised person has seen exactly what will happen and said yes, and human-on-the-loop monitoring lets someone intervene in everything else. This guide covers where to place approval gates by risk tier, how to enforce and bind approvals outside the model, step-up approval and separation of duties for approvers, approval fatigue and automation bias, the audit evidence to keep, and where regulation requires human oversight. It is for security architects, agent platform teams and governance leads.

Key takeaways

Gate by consequence, not by habit. MITRE ATLAS AML.M0029 says Human In-the-Loop policies "should follow the degree of consequence of the task at hand". Reads rarely need a prompt; deletes, transfers, publication and privilege changes do.
Enforce the gate where the agent cannot reach it. OWASP LLM06:2025 allows approval "in a downstream system (outside the scope of the LLM application)" or in the extension itself, never in the agent's instructions alone.
Bind every approval to one pending request. An approval must come from an authenticated human, match a specific pending call with its exact arguments, and be usable once. Unbound or replayable approvals can be forged.
Design against fatigue. Anthropic describes approval fatigue "where people stop paying close attention to what they're approving". Fewer, better prompts with the raw action shown beat more prompts.
Some oversight is a legal duty. The EU AI Act requires high-risk AI systems to be designed for effective human oversight, including the ability to override and stop them, and deployers must assign oversight to people with "the necessary competence, training and authority".

In the loop, on the loop and out of the loop

The terms are used loosely. This guide uses them as practitioner terms, since no standard defines them for agents:

Mode	What the human does	Suits	Typical failure
Human in the loop	Approves or rejects a specific action before it executes; the agent blocks until then	Irreversible, high-value or externally visible actions	Fatigue and rubber-stamping; workflows stall when approvers are absent
Human on the loop	Monitors the agent while it acts, with authority and tooling to pause, override or stop it	High-volume, reversible actions within a bounded scope	Alert seen but nobody stops the agent; no pre-agreed stop criteria
Human out of the loop	Reviews outcomes after the fact (sampling, reports, audits)	Low-risk, read-only or easily reversed tasks	Harm is found only after it has spread

Most production agents need all three, applied per action. ATLAS AML.M0029 describes the in-the-loop control: "Systems should require the user or another human stakeholder to approve AI agent actions before the agent takes them." It adds that tools such as dedicated audit agents "may assist human approval, but final adjudication should be conducted by a human decision-maker". That matters as classifier-based approval spreads: Anthropic's Claude Code auto mode, for example, "delegates approvals to model-based classifiers". A classifier can reduce the number of prompts, but by ATLAS's standard it supports a human decision for consequential actions rather than replacing it.

Where approval gates belong: risk tiers

The OWASP Top 10 for Agentic Applications 2026 names the actions to gate across several risks: ASI01 Agent Goal Hijack calls for "requiring human approval for high-impact or goal-changing actions"; ASI02 Tool Misuse and Exploitation for "human confirmation for high-impact or destructive actions (delete, transfer, publish)"; ASI03 Identity and Privilege Abuse to "Require human approval for high-privilege or irreversible actions"; and ASI09 Human-Agent Trust Exploitation for approval "before accessing extra sensitive data or performing risky actions". ATLAS AML.M0030 adds a context trigger: consider "requiring user confirmation once untrusted data enters the LLM's context".

Tier	Example actions	Gate	Approver
T0 Observe	Read internal documentation, search, summarise, draft	None; logged, sampled after the fact	Not applicable
T1 Reversible change	Edit a file in a branch, create a ticket, update a test record	On the loop; rate and scope limits; gate when untrusted content is in context	Agent owner monitors
T2 Significant change	Merge to main, send an external email, bulk read of personal data, call a new MCP server	Per-action approval with the exact arguments shown	The requesting user or a designated reviewer
T3 Irreversible or privileged	Delete production data, move money, grant access, change the agent's own permissions or goals	Step-up authentication plus approval; dry-run diff first	An approver independent of the requester; two people for the highest impact

Decision guidance

Reversibility first: if an action can be undone in minutes with no external effect, prefer on-the-loop monitoring; if not, gate it. Making actions reversible (soft deletes, staged sends, branch-only writes) moves them down a tier.
Who the agent acts for: a coding assistant acting for one developer can ask that developer; an autonomous agent acting for the organisation needs a named reviewer queue with cover outside working hours.
Third-party tools: the MCP specification says "there SHOULD always be a human in the loop with the ability to deny tool invocations", and OpenAI's Agent Builder safety guidance tells builders to "always enable tool approvals" for MCP tools, reads and writes included. Start strict for new connectors and relax with evidence. See the MCP Security Guide.
Small versus large organisations: a small team can make the requesting user the approver for T2 and a second engineer for T3. Larger organisations should route T3 through an approval service tied to the identity provider, with reviewer groups per system. Security automation follows the same tiers; see the SOAR and Security Automation Guide and the AI SOC Guide.

Enforcing and binding approvals

The AI Agent Authorisation Guide establishes that approval is an authorisation control. The implementation detail decides whether it holds. Agent frameworks now ship approval primitives, and their documentation is candid about the risks:

Mechanism (example)	How it works	Documented caveat
OpenAI Agents SDK `needs_approval`	Set per tool, as a boolean or a per-call function; the run pauses with `interruptions` and resumes after `state.approve()` or `state.reject()`	Snapshots are not authenticated: keep the full snapshot on the server, authenticate and authorise the reviewer, and prevent "the same snapshot from being resumed twice". Sticky decisions for shell and file-edit tools "are not agent-scoped"
Microsoft Agent Framework `approval_mode="always_require"` (.NET: `ApprovalRequiredAIFunction`)	Tool call becomes an approval request; the decision comes from the application, not the agent	By default a response only authorises a call that matches a pending request in the same session; disabling that binding means a "fabricated or replayed response could approve a privileged tool"
Claude Code permission modes and rules	Rules evaluated "deny, then ask, then allow"; modes include `plan`, `acceptEdits`, `dontAsk` and `bypassPermissions`; managed settings can disable bypass and auto modes	Bypass mode is documented for isolated environments such as containers or VMs only; command-matching rules are not "a security boundary around the program"
MCP client confirmation	Clients SHOULD present confirmation prompts and "Show tool inputs to the user before calling the server"	These are SHOULDs; the protocol does not mandate a user interaction model, so check each client

Whatever the framework, five properties make an approval trustworthy:

Out-of-band decision: the approval arrives through the application or an approval service, never as text the model produces or reads from its context.
Authenticated approver: the reviewer signs in to the approval surface with their own identity, and is authorised for that tier and system.
Bound to the request: the decision references a server-held pending item with a hash of the tool name and exact arguments; if the agent changes an argument, the approval no longer matches.
Single use and time-limited: an approval expires and cannot be replayed for a second call.
Scoped persistence: "always allow" choices are scoped to one agent, one tool and narrow arguments, expire, and are reviewed like any other standing permission.

What the approver must see

An approver who sees only the agent's summary is approving the agent's story. OWASP ASI09 Human-Agent Trust Exploitation describes agents that persuade people, and recommends making risk visible "using cues such as red borders, banners, or confirmation prompts". ASI02 recommends to "Display a pre-execution plan or dry-run diff before final approval".

The raw action: tool name, target resource and exact arguments, as the MCP specification puts it, so as "to avoid malicious or accidental data exfiltration".
The effect: a dry-run diff, row counts, recipients, amounts, or the permission delta.
The provenance: which user or event started the task, and whether untrusted content (email, web page, ticket, retrieved document) is in the agent's context.
The risk tier and reason for the gate, plus a clear reject option that does not need justification.

Step-up approval and separation of duties

No agent-specific standard defines approver separation of duties; established access control principles apply. NIST SP 800-53 AC-5 Separation of Duties and CSF 2.0 PR.AA-05 (least privilege and separation of duties) cover who may approve, and IA-11 Re-authentication covers step-up.

Approver is not the requester for T3: the person who asked the agent to act, the agent's developer and the identity the agent acts as should not approve its irreversible actions alone. See the Segregation of Duties Guide.
Step-up at the moment of approval: require phishing-resistant re-authentication for T3 approvals, so a hijacked session cannot approve.
Two-person rule for the highest impact: the EU AI Act sets a precedent in Article 14(5): for certain remote biometric identification systems, no action is taken on an identification unless at least two competent, trained and authorised people have separately verified it.
Grant privilege just in time: pair approval with time-bound elevation, so the agent receives the permission only for the approved action. See the JIT Access Guide.
Agents never approve agents for T3: in multi-agent systems, a supervisor agent may pre-screen, but a human adjudicates.

Human-on-the-loop monitoring and escalation

NIST AI RMF 1.0 frames oversight as an organisational process: GOVERN 3.2 calls for policies that "define and differentiate roles and responsibilities for human-AI configurations", MAP 3.5 for documented processes for human oversight, MANAGE 4.1 for post-deployment monitoring plans, and MANAGE 2.4 for mechanisms "to supersede, disengage, or deactivate" AI systems. In practice, on-the-loop oversight needs:

A live view per agent: current task, recent tool calls, targets touched and spend, with baselines.
Escalation rules written in advance: what an on-call reviewer must do when a threshold trips (pause, gate everything, stop), so stopping is a procedure, not a judgement made under pressure.
Authority to stop: the people on the loop need the access and the mandate to pause the agent. In the OpenAI and Hugging Face breach, as summarised on our breach page, monitoring flagged activity but on-call staff decided not to stop the evaluation.
Dynamic tightening: on-the-loop agents should drop to in-the-loop when signals change, such as untrusted content entering context or a new target outside the task.

Logging detail and the incident runbook are in the AI Agent Observability and Incident Response Guide.

Approval fatigue and automation bias

Anthropic, explaining why it built a classifier-based mode for Claude Code, describes the problem: "Over time that leads to approval fatigue, where people stop paying close attention to what they're approving." The EU AI Act requires overseers of high-risk systems to remain aware of "the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system", which it calls automation bias. The UK ICO, on solely automated decisions under UK GDPR Article 22, says human involvement "has to be active and not just a token gesture". The same pressure that produced MFA fatigue attacks, as in the Uber breach, applies to agent approvals: a stream of prompts trains people to say yes.

Cut prompt volume at the source: remove T0 actions from gating, sandbox reversible work, and use per-call conditions (for example, approve only when the target is production).
Batch only like with like: a single approval for 200 identical ticket updates is reasonable; a batch that mixes a delete with routine edits hides the delete.
Measure attention: track time to decision and approval rate per reviewer; near-instant approvals and approval rates near 100% signal rubber-stamping.
Seed test cases: occasionally present a known-bad request in non-production or training runs and record whether reviewers catch it.
Rotate and cap reviewer load so no one person handles hundreds of approvals a day.

Where regulation requires human oversight

Source	Applies to	Oversight requirement
EU AI Act Article 14	Providers of high-risk AI systems	Systems designed so natural persons can oversee them effectively; measures commensurate with risk, autonomy and context; overseers able to understand limits, watch for automation bias, interpret output, decide not to use or override it, and interrupt it "through a 'stop' button" or similar
EU AI Act Article 26	Deployers of high-risk AI systems	Assign oversight to people with competence, training and authority (26(2)); monitor and suspend use where a risk is suspected (26(5)); keep automatically generated logs for at least six months unless other law says otherwise (26(6))
UK GDPR and GDPR Article 22	Solely automated decisions with legal or similarly significant effects	ICO test: whether a human reviews the decision before it is applied and has discretion to alter it; ICO guidance is under review following the Data (Use and Access) Act 2025
NIST AI RMF 1.0	Voluntary, any AI system	GOVERN 3.2, MAP 3.5, MANAGE 2.4 and MANAGE 4.1 as above

A general-purpose agent is not automatically high-risk under the AI Act; it depends on the use case. Application dates for high-risk obligations have been subject to the EU's digital omnibus process, so check current status in the EU AI Act Compliance Guide.

Audit evidence

An approval that cannot be reconstructed did not happen as far as an auditor is concerned. Map the record to NIST SP 800-53 AU-2 Event Logging, AU-3 Content of Audit Records and AU-10 Non-repudiation, and keep it where the agent cannot edit it (AU-9). Practitioner minimum per approval:

Request: request ID, agent identity and version, the user or event that started the task, risk tier.
What was shown: tool, exact arguments and the diff or preview displayed to the reviewer, stored as rendered.
Decision: approver identity, authentication strength (step-up or not), decision, comment, timestamp and time to decide.
Outcome: the resulting action ID in the target system, so approval and execution can be matched, and any execution without a matching approval flagged.

Rolling out approval gates

Inventory actions, not agents: list each agent's tools and classify every action into T0 to T3.
Shadow mode: log "would have prompted" events for two to four weeks to size the approval load before enforcing.
Enforce T3 first: irreversible and privileged actions get bound, step-up approvals in the platform or gateway.
Pilot T2 with one team: measure approval rate, time to decision and abandoned tasks; tune conditions.
Disable bypass paths: use managed settings or policy to block bypass modes and blanket "always allow" in production.
Roll back by tier: if gates stall critical work, relax T2 conditions while keeping T3 enforced; never roll back by switching approvals off globally.
Test adversarially: include prompt-injection attempts to get an action past the gate. See Red Teaming AI Agents for Identity Abuse.

Measuring whether oversight works

Gate coverage: percentage of T2 and T3 actions executed with a matching, bound approval; the target is 100%.
Unmatched executions: count of high-impact actions in target-system logs with no approval record.
Approval rate and time to decide per reviewer, to detect rubber-stamping.
Prompt volume per agent per day, the leading indicator of fatigue.
Seeded-case catch rate: share of known-bad test requests rejected.
Stop latency on the loop: time from monitoring alert to pause decision.

Framework mapping

Control	Mapping
Approval before agent actions	MITRE ATLAS AML.M0029; OWASP LLM06:2025; OWASP ASI01, ASI02, ASI03, ASI09
Gate on untrusted context	ATLAS AML.M0030 (counters AML.T0051 LLM Prompt Injection leading to AML.T0053 AI Agent Tool Invocation)
Approver separation and step-up	NIST SP 800-53 AC-5, IA-11; CSF 2.0 PR.AA-05; ISO/IEC 27001:2022 A.5.3 Segregation of duties
Oversight roles	CSF 2.0 GV.RR-02; NIST AI RMF GOVERN 3.2; EU AI Act Art. 26(2)
Approval evidence	NIST SP 800-53 AU-2, AU-3, AU-9, AU-10; ISO/IEC 27001:2022 A.8.15 Logging
Design for oversight	EU AI Act Art. 14; NIST SP 800-53 SA-8(27) Human Factored Security

Common failure patterns

Approval by instruction. Telling the agent to ask first is not a gate. In the Replit agent database deletion, a code freeze stated in instructions did not stop the agent deleting production data.
No gate at the platform. In the PocketOS database deletion, an agent with an account-scoped token deleted a production volume through an API call that asked nobody.
Sticky approvals that spread. An "always allow" for one agent or command family silently covers others.
Forgeable approvals. Responses not bound to a pending request, or snapshots resumable twice, let a replayed or injected approval through.
Summary instead of action. Reviewers approve the agent's description, not the arguments that will execute.
On the loop without authority. Monitoring fires, but reviewers lack the mandate or tooling to stop the agent.

Practitioner checklist

Classify every agent action into risk tiers and gate T2 and T3 actions.
Enforce approvals in the platform or gateway, never only in the agent's instructions.
Bind each approval to a server-held pending request with exact arguments, single use and expiry.
Show reviewers the raw action, a dry-run diff and the provenance of untrusted content.
Require step-up authentication and an independent approver for irreversible or privileged actions.
Disable bypass modes and unscoped "always allow" in production through managed policy.
Write on-the-loop escalation rules and give reviewers the authority to pause agents.
Log what was shown, who decided and the resulting action, and alert on unmatched executions.
Track approval rates, decision times and seeded-case catch rates to detect fatigue.
Check whether any agent use case falls under EU AI Act high-risk or Article 22 obligations.

Human-in-the-Loop for AI Agents Guide: Approval Gates and Oversight Patterns