Yes, for actions that create external effects or expose data. Confirmation should be mandatory before sending messages, deleting records, accessing private data, or controlling connected devices. The goal is not to slow every interaction, but to stop silent escalation when the assistant is steering from retrieved content rather than explicit intent.
When should user confirmation become mandatory?
Confirmation should be required whenever an AI action can create an external effect, reveal private information, or cross a trust boundary. That includes sending messages, changing records, deleting data, spending money, invoking APIs, or controlling connected devices. The key test is whether the action would be hard to undo, hard to detect, or likely to surprise the user if it happened silently.
That standard is stronger than “important action” and weaker than “confirm everything.” A good policy focuses on user-visible consequences, not every intermediate step. If the assistant is only drafting, ranking, or summarising, confirmation usually adds friction without reducing much risk. If the assistant is about to act on behalf of the user, confirmation becomes part of the control surface.
For teams that want a practical reference point, the human-and-machine boundary issues in Human vs Non-Human Identity are directly relevant, because the same governance problem appears when software is allowed to operate with delegated user authority.
Which AI actions should always be treated as sensitive?
The highest-risk actions are the ones that can leak data, commit the organisation, or move the environment into a new state. Sending external email, posting to chat systems, approving transactions, deleting or overwriting records, exporting private content, granting access, and issuing commands to devices all fit that pattern. The more the action resembles a real-world decision rather than a suggestion, the stronger the case for explicit confirmation.
Context matters, too. A message drafted for internal review is not the same as a message sent to a customer. A retrieved answer shown on screen is not the same as a record update in a live system. The confirmation boundary should follow the action that creates persistence, disclosure, or authority, not the model call that produced the draft.
This is also why OWASP ASVS is useful as a design reference: it reinforces that authentication, access control, and action validation are distinct checks, not one generic safety step.
How should confirmation be designed so it helps rather than annoys users?
The best confirmation prompts are specific, contextual, and binding. They should state what will happen, to whom, in which system, and with what scope. “Send message to external customer” is better than “Are you sure?” because the user can evaluate the consequence. If the action is irreversible or highly sensitive, the UI should force a deliberate decision rather than a passive timeout.
Confirmation should also be selective. If every low-impact action needs a prompt, users will approve blindly and the control will fail in practice. If only high-impact actions trigger confirmation, users stay alert when it matters. That is the right trade-off: fewer prompts, but with much higher signal value.
For action-driven systems and delegated workflows, the same principle shows up in OWASP Agentic AI Top 10, which treats identity and privilege abuse as a distinct design problem, not just a UX concern.
Risk and Threat Considerations
Unchecked AI actions create a silent-escalation problem: the system can move from assisting the user to acting for the user without a clear checkpoint. That raises the chance of unintended disclosure, malicious prompt manipulation, or harmful actions triggered by misleading retrieved content, stale context, or overbroad permissions.
Failure mechanism: The assistant is allowed to execute a high-impact action without an explicit human decision at the moment of execution, so the user loses the last meaningful opportunity to notice scope, destination, or side effects.
Impact: Data can be exposed, records can be changed, messages can be sent, or devices can be controlled in ways the user did not intend, and the resulting action may be hard to reverse or even detect quickly.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and OWASP API Security Top 10 address the attack and risk surface, while NIST SP 800-53 Rev 5 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Covers AI actions crossing delegated authority and privilege boundaries. |
| Recommendation — Require explicit human confirmation before agents execute privileged or externally visible actions. | ||
| OWASP API Security Top 10 | API5 — Broken Function Level Authorization | Sensitive AI actions often invoke functions that must be separately authorised. |
| Recommendation — Gate high-impact functions behind explicit authorization checks before execution. | ||
| NIST SP 800-53 Rev 5 | AC-6 — Least Privilege | Confirms only the minimum authority needed for AI-triggered actions. |
| IA-5 — Authenticator Management | Sensitive actions depend on strong control of credentials used to perform them. | |
| Recommendation — Limit AI-triggered permissions to the smallest set needed for the approved task. Protect and rotate the credentials that can execute sensitive AI actions. | ||
| NIST CSF 2.0 | PR.AA-05 — Least Privilege | Maps to ensuring AI actions only proceed with the minimum necessary access. |
| Recommendation — Enforce least-privilege access for any AI workflow that can affect external systems. | ||
Practitioner Guidance
What to prioritise: Classify actions by external effect first, then by sensitivity. If an action changes state outside the assistant, treat it as a candidate for mandatory confirmation unless it is clearly low-risk and immediately reversible.
What to verify: Confirm that the prompt shown to the user names the target, the exact action, and the scope of impact. A generic approval dialog is not enough when the action can disclose data or trigger downstream execution.
Common mistake: Teams often place confirmation on the model output, not on the actual side-effecting operation. That leaves a gap where the system can still execute something the user never explicitly approved.
Practitioner takeaway: Require confirmation at the point where the AI crosses from recommendation into action, and reserve that friction for decisions that can leak data, alter records, or create hard-to-undo external effects.