Why do AI platforms create confused deputy risk in enterprise environments?

Why This Matters for Security Teams

confused deputy risk matters because AI platforms often act with delegated authority, not just decision support. When an orchestrator can read mailboxes, query internal APIs, trigger workflows, or retrieve secrets, it becomes a high-value intermediary that can be steered into legitimate-looking actions. That breaks the normal assumption that the caller and the beneficiary of an action are the same party, which is central to many enterprise controls.

This is especially dangerous in environments that still rely on broad service account trust, shared tokens, or flat tool permissions. Guidance from the NIST Cybersecurity Framework 2.0 emphasizes governance and access control, but AI platforms add a layer of indirection that many control sets were not designed to inspect in detail. NHIMG research on OWASP NHI Top 10 shows how delegated machine access becomes fragile once tool use is chained across systems.

In practice, many security teams encounter confused deputy failures only after an AI workflow has already been induced to access data or execute actions it was never meant to perform.

How It Works in Practice

The core problem is that the platform holds authority on behalf of many users, but downstream services often cannot tell whether a specific action was truly intended, safely scoped, or merely triggered by attacker-controlled content. An AI agent that can parse prompts, call tools, and continue multi-step tasks can be manipulated into using its own privileges against the environment.

That is why static, role-based access alone is not enough. If the platform has a long-lived token or broad service identity, an attacker only needs to get the model to choose the wrong path. Current best practice is shifting toward workload identity, just-in-time credentials, and request-time policy evaluation. In practical terms, the platform should prove what it is, and every tool call should be checked for purpose, context, and scope before execution.

• Bind each agent or workflow to a distinct workload identity instead of a shared integration account.
• Issue short-lived tokens per task, not reusable secrets that survive across sessions.
• Evaluate tool use at runtime with policy-as-code so the action is approved in context, not just by role.
• Pass user intent and task provenance to downstream services so they can distinguish delegated requests from autonomous ones.

NHIMG’s Ultimate Guide to NHIs — Why NHI Security Matters Now and the McKinsey AI platform breach illustrate how platform-level trust can become a blast-radius amplifier when one orchestrator fronts many data sources and automation paths.

These controls tend to break down when a single AI platform brokers access to legacy systems that cannot validate request context or separate user intent from platform authority.

Common Variations and Edge Cases

Tighter delegation controls often increase integration overhead, so organisations must balance stronger containment against the operational cost of more granular identities and policy checks. There is no universal standard for this yet, especially when agents share tools across teams or when a single workflow spans multiple clouds, SaaS platforms, and internal services.

One common edge case is a system that is safe in isolation but unsafe in composition. A model may not have direct access to sensitive data, yet it can ask another tool to fetch it, then route the result into a third system that performs an action the user never explicitly requested. Another issue appears when developers collapse many agents into one service principal for convenience, which makes least privilege impossible to enforce meaningfully. The Top 10 NHI Issues and the OmniGPT breach both reinforce how fast trust boundaries erode once credentials and orchestration are centralised.

Best practice is evolving toward intent-based authorisation, per-action consent for high-risk operations, and explicit separation between the user who requested a task and the platform that executes it. That said, these designs are harder to retrofit into older systems, especially where APIs were built to trust authenticated callers rather than verify why they are acting.

Related resources from NHI Mgmt Group

• Why do AI agents create new risk in non-human identity management?
• When does AI agent access create more risk than it reduces?
• When do AI agent credentials create more risk than they reduce?
• Why do AI agents create more risk when they reuse existing credentials?