Teams should split them when the system does more than proxy prompts and begins making tool decisions, calling MCP servers or acting on behalf of a user. At that point, the edge gateway remains useful, but it is no longer sufficient on its own. The agent layer needs its own identity, authorization and audit controls.
What changes when the gateway is no longer the whole control plane?
The split becomes necessary when prompt forwarding is no longer the full job. A gateway can standardise access to models, enforce request filtering, and centralise logging, but it does not govern what happens after the system starts choosing tools, chaining steps, or acting with delegated authority. That is the point where agent governance becomes a separate control layer.
At that stage, the main design question changes from “which model can this request reach?” to “what is this agent allowed to do, under whose authority, and with what audit trail?” That is a materially different control problem because the risk moves from content handling at the edge to action execution inside the workflow.
The distinction matters for teams that want to avoid mixing transport controls with decision controls. A gateway can stay in place as the outer checkpoint, but once the runtime can invoke tools or MCP servers, governance has to follow the action path, not just the ingress path.
Why tool use, delegation and on-behalf-of actions force a separate layer
When an AI system can select tools or call external services, the security boundary shifts to authorisation and delegation. The system is no longer only passing messages, it is initiating side effects, consuming privileges and creating business-impacting actions. That means the agent layer needs its own identity model, policy enforcement and auditability, even if the edge gateway still handles rate limits, content inspection and tenant separation.
In practice, the split is usually justified when one or more of these conditions appear: the agent can choose among tools, it can act on a user’s behalf, it can access internal systems, or it can hold credentials that outlive a single request. Those are all signs that the control problem has become privilege-bearing, not just request-bearing.
If the system only proxies prompts and returns model output, a gateway may be enough. Once the same system can modify records, trigger workflows, read scoped data, or delegate actions to MCP-connected tools, you need a distinct governance layer for approval, scoping and attribution.
How to draw the boundary without overengineering
The cleanest boundary is to treat the gateway as the ingress control and the agent layer as the execution control. The gateway should answer questions about who may reach the AI service, what traffic is allowed, and what content needs inspection. The agent layer should answer questions about what an agent may do, which tools it may call, which scopes it may assume, and when human approval is required.
AI Agent Authorisation Guide is useful here because it separates per-action policy decisions from the outer request path. That is the right mental model when the control point moves from the prompt to the act.
Agentic AI Identity Guide fits the same boundary problem from the identity side, especially where delegated authority, registration and retirement have to be managed separately from the model endpoint. If the agent can continue operating over time, its identity lifecycle becomes part of the design, not a by-product of the gateway.
AI Agent Observability, Audit and Incident Response Guide becomes relevant once the system can take actions that must be attributable after the fact. The evidence problem is different from normal API logging because you need to reconstruct intent, tool choice and execution sequence, not just request volume.
Risk and Threat Considerations
Once tool use and delegated actions enter the design, the main risk is overtrusting the gateway as if it were the whole defence. That creates a gap where a well-filtered request can still produce a harmful action, because the actual abuse happens after the request crosses the boundary.
Failure mechanism: The system grants broad or persistent execution rights to the agent, then relies on edge filtering instead of per-action policy, so a compromised prompt, malicious instruction or mistaken tool selection can produce unauthorised side effects.
Impact: Attackers or misconfigured workflows can trigger data access, workflow abuse, credential use or destructive operations while leaving only partial evidence at the gateway, which makes containment and attribution harder.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST SP 800-53 Rev 5 sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Agent actions and delegated authority are central once the system can call tools. |
| Recommendation — Enforce per-action privilege checks and approval gates for agent execution. | ||
| NIST SP 800-53 Rev 5 | IA-9 — Service Identification and Authentication | Tool-calling agents and MCP-connected services need distinct machine-to-machine trust. |
| AU-6 — Audit Record Review, Analysis, and Reporting | Agent decisions and tool actions need reviewable records beyond gateway logs. | |
| AC-6 — Least Privilege | Separate agent governance exists to stop overbroad tool and data access. | |
| Recommendation — Require service authentication for agent-to-tool interactions and session establishment. Collect and review agent action logs that preserve decision and tool-use context. Limit each agent to the minimum tool scopes and data paths it actually needs. | ||
| OWASP Non-Human Identity Top 10 | NHI-05 — Overprivileged NHI | Agent identities that can act on behalf of users should not carry excessive standing privilege. |
| Recommendation — Reduce standing privilege for agent credentials and scope access to specific tasks. | ||
Practitioner Guidance
What to prioritise: Split the layers when the agent can do more than answer. If the runtime can call tools, reach internal systems or act for a user, define a separate control plane for authorisation, identity and audit before expanding the tool set.
What to verify: Check whether the gateway is only enforcing ingress policy or whether it is accidentally being used to make downstream permission decisions. If the same control is expected to cover request screening and action approval, the design is already too coarse.
Practitioner takeaway: The gateway should guard entry, but the agent layer must govern action. The moment the system can make decisions that change state, authority and traceability have to be designed as first-class controls, not assumed to follow from the model front door.