What breaks when runtime approval is only described in a prompt?

Prompt-based approval can be ignored, bypassed, or misread by the model, especially when context changes or a tool is retried. A control that matters must be enforced by the runtime, with a clear decision path and logs for every attempted action.

Why This Matters for Security Teams

When runtime approval exists only as a prompt instruction, it is not a control boundary. The model can reinterpret, compress, ignore, or lose that instruction as conversation state changes, which means the approval step is no longer tied to the action itself. Security teams should treat this as an enforcement gap, not a usability issue. NIST guidance in the NIST Cybersecurity Framework 2.0 emphasizes governance, control execution, and continuous monitoring, all of which are undermined when approval is left to prompt text alone.

The practical risk is that a tool call, file write, payment action, or administrative change may proceed after a model decides the prompt is ambiguous, stale, or outweighed by a newer instruction. In agentic workflows, that failure mode is amplified because the agent may retry operations, chain tools, or operate across multiple turns. The result is an approval process that looks present in design but does not exist at execution time. In practice, many security teams encounter this only after an agent has already acted outside its intended scope, rather than through intentional testing.

How It Works in Practice

Runtime approval must be enforced outside the model, typically by the orchestration layer, tool gateway, or policy engine. The model may propose an action, but the runtime decides whether the action can execute. That means the approval decision should be based on identity, context, tool type, data sensitivity, and current risk state, not on whether the prompt contains a polite request for confirmation.

Operationally, a sound design separates intention from execution:

• The agent drafts an action request.
• The runtime evaluates policy before any tool is called.
• Human approval, if required, is recorded as a discrete decision.
• The tool execution path is logged with timestamps, actor identity, and request context.
• Retries and chained actions are re-evaluated, not assumed safe because an earlier step was approved.

This is where identity governance matters. If the agent has access to secrets, privileged endpoints, or non-human identities, the runtime should bind approval to the specific identity and operation, not to the conversational thread. Controls aligned to zero trust and least privilege are stronger than prompt-only instructions because they are enforced independently of model behavior. OWASP’s guidance on OWASP Top 10 for Large Language Model Applications is useful here, especially where prompt injection or tool misuse can alter the execution path. MITRE ATLAS also helps security teams think about adversarial manipulation of AI workflows.

These controls tend to break down in multi-agent environments with shared tool access and weak event logging because one agent’s approved action can be replayed, delegated, or expanded by another agent without a fresh policy decision.

Common Variations and Edge Cases

Tighter runtime approval often increases latency and operator workload, requiring organisations to balance fast automation against the risk of silent overreach. Best practice is evolving for high-autonomy systems, and there is no universal standard for how often human approval must be reasserted in long-lived agent sessions.

One edge case is cached approval. If a system treats a prior approval as valid for later, similar-looking actions, the control weakens quickly when context shifts. Another is partial approval, where a reviewer authorises a broad task but the agent decomposes it into multiple tool calls that were never individually reviewed. This is especially dangerous when prompts mention approval but the runtime does not attach policy state to each call. For AI-specific control design, the NIST AI Risk Management Framework is helpful for structuring governance, while OWASP guidance for LLM applications highlights how prompt-layer assumptions fail under injection or tool abuse.

In regulated or safety-sensitive environments, approval should also be paired with immutable audit logs and explicit rollback procedures. Where agents touch financial systems, personal data, or privileged administration, the runtime needs clear deny-by-default behavior. That becomes especially important when a tool is retried after timeout, because the second attempt may be treated as operational noise unless the policy engine rechecks the request. Current guidance suggests that prompt-only approval is acceptable for demonstrations, but not for production control planes.

Related resources from NHI Mgmt Group

• What breaks when whitelist-based prompt approval is used for dynamic agents?
• What breaks when AI runtime attacks are treated as prompt-safety issues only?
• What breaks when MCP governance stops at approval and ignores runtime behaviour?
• What breaks when AI agents are exposed to prompt injection without runtime baselines?