How Kong AI Gateway changes controls for single LLM agents

At a glance

What this is: NHI Mgmt Group's analysis, drawing on Kong's blog post: A single LLM agent built with LangGraph still depends on ordinary API and gateway controls to constrain model calls and external functions.

Why it matters: It matters because IAM, API security, and AI governance teams need to treat agent behaviour as governed runtime access, not as an abstract model-risk problem.

Context

Single LLM agents are software workflows that call a model, invoke tools, and pass state between steps. The security question is not whether the agent is sophisticated, but which credentials, routes, and policies govern each runtime call.

Kong's article frames LangGraph as a simple agent pattern, then shows why the control point shifts to the gateway when the agent starts reaching out to external functions and APIs. That makes the identity problem closer to API authorization and secrets governance than to model tuning alone.

For IAM and security teams, the relevant issue is whether existing controls can still express who or what is allowed to call a model, a tool, or an upstream API when the request originates from software rather than a person.

Key questions

Q: How should security teams govern LLM and agent access in production?

A: Treat every model, agent, connector, and service account as part of one identity surface. Define owners, scope permissions to task and environment, and require logging that shows who or what accessed the system, what data was touched, and what downstream actions occurred.

Q: Why do API keys and gateway policy still matter for AI agents?

A: Because the agent still depends on ordinary machine credentials to reach models and external functions. If those credentials are broad or shared, the agent inherits the blast radius of every permission behind them, and the gateway becomes the only place to meter and constrain that activity.

Q: What breaks when an AI agent shares one credential across tools and model calls?

A: The boundary between inference and action breaks down. A single credential can then authorise model calls, function calls, and downstream API access in the same session, which makes it hard to audit intent, limit scope, or contain misuse when the agent behaves unexpectedly.

Q: How should teams detect when agent access is broader than intended?

A: Look for agents that connect to multiple SaaS tools, re-register repeatedly, or carry scopes that are consistent across very different tasks and tenants. Those patterns usually show that delegation is being reused as a convenience layer rather than governed as a bounded identity state.

Technical breakdown

Why gateway policy is the real control plane for single LLM agents

A single LLM agent may look like application logic, but every external action still passes through a set of access decisions. In Kong's pattern, the gateway sits between the agent and the upstream model or function, which means authentication, authorization, and rate limits are enforced at the point of use rather than inside the prompt or graph. That matters because the agent can only be as safe as the runtime policy that governs model calls, tool invocations, and token consumption. The architecture is not agent-first. It is policy-first, with the agent inheriting the trust boundary of the gateway.

Practical implication: model and tool access should be controlled at the gateway, not left inside the agent code.

How LangGraph state and tool selection change the identity problem

LangGraph represents the agent as nodes, edges, and state, which makes the workflow easy to reason about but also easy to over-trust. The state carries messages and responses, while tools introduce external calls that may depend on separate credentials, scopes, and upstream permissions. That means the security boundary is no longer the model response alone. The boundary now includes which functions the agent can reach, what parameters it can pass, and whether those calls are observable enough for audit. The control issue is less about prompt content and more about whether tool access is scoped to the exact runtime path the graph can take.

Practical implication: map each agent tool to a distinct authorization path and audit it separately from the prompt flow.

Why API keys and rate limits still matter for AI workloads

The article keeps returning to ordinary API controls because the agent still depends on them. API keys identify the caller, while rate limits constrain how often the caller can consume model or tool capacity. In practice, that means the agent is governed like any other non-human client, except the consumption pattern can be more dynamic and less predictable. If credentials are broad, long-lived, or shared across tools, the agent inherits that exposure. If rate limiting is too coarse, the gateway may protect the model but fail to distinguish normal automation from runaway loops or excessive tool use. The mechanism is simple: the agent does not escape IAM just because it is AI-flavoured.

Practical implication: treat agent API keys as high-value NHI credentials and bound their scope, lifetime, and quotas.

Threat narrative

Attacker objective: The objective is to turn a governed agent into an over-privileged caller that can consume model capacity and reach external functions outside its intended scope.

1. Entry occurs when the agent is given an API key and gateway access to a model endpoint and external tools.
2. Escalation happens if the same runtime identity can reach multiple tools or upstream services without tight scope separation.
3. Impact follows when the agent can consume tokens, invoke functions, or call APIs beyond the intended workload boundary.

Breaches seen in the wild

• LiteLLM MCP auth bypass 2026: An exploited LiteLLM MCP auth bypass and default sk-1234 master keys let attackers steal AI gateway master and provider API keys.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Single LLM agents do not create a new identity category, they inherit the risk profile of the credentials and gateways that let them act. Kong's article is useful because it strips away the AI language and exposes the underlying governance truth: an agent is only as constrained as the API key, route policy, and tool scope behind it. For practitioners, the central question is whether the agent is merely another NHI caller or a hidden privilege concentration point.

Gateway enforcement becomes the decisive control when agents can select when to call a model and when to invoke external functions. The article's LangGraph example shows that the runtime decision point sits in the workflow, not in a static application role. That means policy has to follow the call path, not the code repository. Practitioners should read this as a signal that agent safety is a runtime authorization problem, not a prompt-engineering problem.

Identity blast radius is the right concept for single LLM agents that touch both models and tools. Once one agent key can reach multiple upstream services, the security model stops being about isolated access and becomes about shared failure domains. That is a governance issue because token sprawl, tool sprawl, and gateway sprawl compound each other. Teams should expect the blast radius of a single agent credential to become the real unit of risk.

LangGraph makes control boundaries visible, but it does not remove the need for NHI discipline. Nodes, edges, and state improve traceability, yet the agent still depends on secrets, authorisation, and metering. The discipline that matters is OWASP-NHI plus API governance, because those are the layers that decide whether a machine caller is well scoped or over-extended.

Single LLM agents are a proving ground for whether existing IAM programmes can govern software that thinks in runtime steps. If the programme can only classify the model but not the caller, the tool, and the gateway policy together, it is not yet governing the agent. The implication is straightforward: identity teams need a control model that treats agent execution as a first-class machine identity workflow.

From our research library:

• 71% of organizations use third-party APIs, according to Gartner’s 2024 data.
• Read next: AI Agent Identity Security Buyer's Guide

What this signals

Identity blast radius: single LLM agents concentrate model access, tool access, and API access into one runtime path, so the practical risk is not model sophistication but over-broad machine authority. Teams that already govern service accounts should recognise the same pattern here, only with more dynamic call behaviour and more places for scope drift to hide.

Kong's article is a reminder that agent governance starts where the gateway sees the request. If security teams cannot distinguish model consumption from tool invocation, they will struggle to enforce least privilege or produce a meaningful audit trail for software that decides its own execution path.

For practitioners

• Define the agent as a machine caller Assign each single LLM agent its own non-human identity, separate from developer, service, and platform credentials. That identity should be traceable to one workflow and one gateway policy set.
• Scope every tool to a distinct policy path Map model calls, function calls, and external API calls to separate authorization decisions so one agent credential cannot silently cross from inference into unrestricted tool use.
• Limit token and request consumption per agent Apply quotas that distinguish normal agent operation from runaway loops, repeated retries, or chained tool use that can exhaust upstream model budgets.
• Separate model access from tool access Do not assume access to a model endpoint implies access to downstream functions. Use gateway policy to keep inference permissions and tool permissions independently governed.
• Log the full agent execution path Capture the prompt, tool selection, upstream API target, and response so security and audit teams can reconstruct what the agent was authorised to do at each step.

Key takeaways

• Single LLM agents still rely on ordinary machine credentials and gateway policy, so IAM and API controls remain the core governance layer.
• The main control risk is not the agent's language ability but its runtime access path to models, tools, and external APIs.
• Teams should separate identities, scopes, and quotas for agent execution so one credential does not become a broad machine privilege bundle.

Key terms

• Single LLM Agent: A single LLM agent is a software workflow that uses one language model to decide and execute a bounded sequence of actions. In governance terms, it is still a machine caller, so its access must be controlled through credentials, policy, and audit just like any other non-human identity.
• Gateway Policy Enforcement: Gateway policy enforcement is the use of an ingress control point to apply security and governance rules before traffic reaches its destination. It matters in event-driven systems because it lets teams centralise authentication, schema checks, throttling, and logging for both APIs and brokers.
• Runtime Authorisation: Runtime authorisation is the practice of deciding access while a task is in progress, rather than only at provisioning time. It matters for NHIs because credentials and entitlements can change risk mid-session, especially when automation or AI agents interact with sensitive systems.
• Identity Blast Radius: The amount of damage a compromised identity can cause across systems, data, and infrastructure. In NHI environments, it is shaped by permissions, network reach, and administrative capability rather than by the credential alone. Reducing blast radius is a containment strategy that limits lateral movement and data exposure.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.