What happens when an LLM agent is manipulated into acting as a Confused Deputy?

How a Confused Deputy Changes What an LLM Agent Can Do

An LLM agent becomes dangerous in a Confused Deputy setup when its legitimate authority is redirected toward the attacker’s goal. The model is not “hacked” in the classic sense, but it is induced to use valid permissions in an invalid way. That matters because downstream systems often trust the agent’s authenticated identity more than the caller’s intent.

In practice, the attack succeeds when the agent can reach actions that are broader than the original request really requires. If the agent can initiate refunds, retrieve records, change settings, or call privileged tools, the attacker may only need to shape the prompt or surrounding context to steer those capabilities. The result is an action that looks authorized to the platform even though the business intent is not.

The key failure is authority mismatch. The user or attacker provides the instruction, but the agent supplies the privilege. When those two are not tightly bound, the system can no longer tell whether the action was a legitimate delegation, a mistaken execution, or abuse of an assumed-trusted workflow.

Why Confused Deputy Attacks on Agents Are Hard to Spot

These failures are often subtle because the output may be operationally valid. A refund can post, a record can be disclosed, or a change can be applied without triggering an obvious error. That makes the incident look like normal automation unless the system preserves enough context to distinguish user intent from delegated authority.

For agentic workflows, the risk is amplified by tool use, token passthrough, and shared sessions. An attacker does not need to own the agent’s account if they can cause the agent to borrow that account’s standing rights on their behalf. The most dangerous designs are the ones that let an agent act broadly while giving weak visibility into who benefited from the action.

Identity boundaries matter here because delegation must be explicit, scoped, and auditable. Where an agent can act on behalf of multiple users or systems, the safer pattern is to bind authority to a specific request, resource, and session context rather than to a general bearer of trust. NHIMG’s Confused Deputy in AI Agents Guide covers the core delegation controls that prevent this class of abuse.

Controls That Reduce Delegation Abuse Without Breaking the Agent

Good defense starts by separating what the agent may do from what the requester asked it to do. That means narrow scopes, explicit consent boundaries, resource-specific authorization, and checks that link the action to the intended beneficiary. A generic “agent is allowed” policy is not enough when the request can be repurposed mid-flight.

Session design also matters. If the agent can reuse a powerful credential across tasks, compromise becomes much cheaper and harder to contain. Use short-lived, task-scoped credentials where possible, and make sure the agent cannot freely swap context between users, tenants, or tools.

For cloud and platform integrations, trusted relationship controls deserve special attention. NHIMG’s Cloud Workload Identity Guide explains why confused deputy failures often arise when identity federation or role assumption is too broad, while the AWS STS and AssumeRole Guide shows how to constrain delegation with conditions such as external IDs, source identity, and session controls.

Risk and Threat Considerations

Confused deputy abuse is risky because it turns trusted automation into an authorization bypass. The attacker is not trying to defeat the agent’s permissions directly, but to piggyback on them, which can produce fraud, data disclosure, or unauthorized state changes that blend into ordinary system activity.

Failure mechanism: The system grants the agent broad authority, then fails to bind each action to the true requester, intended resource, and allowed delegation path.

Impact: Attackers can trigger valid-looking actions that the platform attributes to the agent, not to the malicious prompt or caller, which expands blast radius and weakens accountability.

Practitioner Guidance

What to verify: Check whether the agent can exercise privileges that exceed the minimum needed for the current task. If the answer is yes, treat the design as delegation-heavy and confirm that the action path is constrained by resource, audience, and session context rather than by a general trust relationship.

Decision rule: If a request can cause money movement, record access, or entitlement changes, require an explicit authorization boundary that survives prompt manipulation. If that boundary cannot be enforced, reduce the agent’s authority before expanding its tool set.

Practitioner takeaway: The objective is not to stop agents from acting on behalf of users, it is to make sure the agent cannot be tricked into using real privilege for the wrong beneficiary.

Related resources from NHI Mgmt Group

• What is the difference between human identity governance and AI agent governance?
• When does AI agent access create more risk than it reduces?
• What is the difference between governing human access and governing AI agent access?
• When do AI agent credentials create more risk than they reduce?