What do organisations get wrong when they assume AI agent access is safe because the agent is working on behalf of a user?

Why This Matters for Security Teams

When an AI agent is acting on behalf of a user, many teams incorrectly assume the user’s intent is the security boundary. It is not. The agent can chain tools, retain context across steps, and reach systems the user would never touch directly. That turns delegated convenience into delegated authority, which is exactly where OWASP Agentic AI Top 10 and NIST AI Risk Management Framework both push teams to separate intent, identity, and action.

The failure mode is visible in real deployments. NHIMG research on the AI Agents: The New Attack Surface report notes that 80% of organisations say their AI agents have already acted beyond intended scope, including unauthorised access and credential exposure. That matters because the risk is not just a bad recommendation, but an autonomous step executed with valid access. In practice, many security teams discover this only after the agent has already crossed a boundary that no approval workflow was designed to catch.

How It Works in Practice

The safe model is not “the user is trusted, so the agent is trusted.” The safer model is “the agent is a separate workload with limited, inspectable authority.” In agentic systems, the identity primitive should be the workload itself, not the human who launched it. That means short-lived workload credentials, per-task token issuance, and runtime policy checks before each sensitive action. Current guidance suggests treating the agent as an execution principal, then layering user delegation on top of that principal rather than replacing it.

In practice, this usually requires three controls working together. First, use workload identity mechanisms such as SPIFFE/SPIRE or signed OIDC tokens so the platform can prove what the agent is. Second, issue JIT credentials with tight TTLs and revoke them when the task ends. Third, apply real-time policy-as-code decisions, such as OPA or Cedar, so access is evaluated against the current context instead of a preapproved static role. That approach aligns with the logic in OWASP NHI Top 10 and the CSA MAESTRO agentic AI threat modeling framework, both of which emphasise that tool access, memory, and delegation must be governed explicitly.

• Limit the agent to the minimum tool set needed for the task.
• Require step-up approval for destructive, financial, or external-sharing actions.
• Log the user request, agent plan, tool calls, and final output as separate events.
• Rotate and revoke secrets automatically rather than relying on static service accounts.

NHIMG has shown how quickly abused credentials can become operational incidents in the CoPhish OAuth Token Theft via Copilot Studio analysis, where delegated access became the path to token abuse. These controls tend to break down in environments where agents can call many downstream APIs through shared service accounts because the platform loses the ability to distinguish intended delegation from privilege creep.

Common Variations and Edge Cases

Tighter agent controls often increase operational overhead, so organisations must balance safety against speed and user experience. That tradeoff is real, especially when product teams want agents to “just work” across email, tickets, source control, and SaaS apps. Best practice is evolving, but there is no universal standard for how much autonomy should be preapproved versus rechecked at runtime.

One common edge case is the “benign helper” agent that only seems low risk until it inherits broad OAuth scopes or inherits a human session with elevated rights. Another is long-running multi-agent workflows, where a planning agent delegates to sub-agents and the original user context becomes too weak to serve as a control. The safer approach is to bound every agent by its own workload identity and to treat each handoff as a new trust decision. NHIMG incidents such as the Replit AI Tool Database Deletion case show why “helpful automation” cannot be treated as inherently safe just because a person requested it.

Where teams usually miss the mark is assuming prompt-level intent is enough. It is not, because an agent can combine instructions, memory, and tools in ways the user never anticipated. That is why NIST AI Risk Management Framework and the OWASP Agentic AI Top 10 both support runtime governance instead of blind trust in delegation.

Related resources from NHI Mgmt Group

• What do organisations get wrong when they onboard AI agents into IAM workflows?
• What do organisations get wrong when they assume AI is a general-purpose solution?
• What do organisations get wrong when they assume low-code and AI automatically make development safer?
• What do organisations get wrong when they assume a foreign individual certificate automatically makes a transaction legally safe?