Single LLM agents on Kong AI Gateway: what changes for IAM teams?

TL;DR: LangGraph-based agents still depend on API keys, tool access, and gateway policy to control model calls and external functions, making identity and rate limiting central to agent safety, according to Kong. The real issue is not agent complexity but whether existing IAM and API controls can govern runtime access decisions.

NHIMG editorial — based on content published by Kong: How to Build a Single LLM AI Agent with Kong AI Gateway and LangGraph

Questions worth separating out

Q: How should security teams govern tool calling in AI agents?

A: Security teams should govern tool calling as a privileged access problem, not as a developer convenience.

Q: Why do AI agents create new identity governance risks for IAM teams?

A: AI agents create new governance risk because they combine identity, policy, and runtime decision-making in one execution path.

Q: What breaks when agent access is reviewed only at provisioning time?

A: Provisioning-time review misses the real decision point, which is the moment the agent selects a tool or sends a request to an external service.

Practitioner guidance

• Inventory every agent-to-tool trust path Map where each agent authenticates, which tools it can call, and which credentials are used at each hop.
• Separate model access from tool authority Do not assume that a user allowed to query a model should also be allowed to invoke downstream functions through that model.
• Apply rate limits to agent traffic, not just humans Use token-aware throttling and policy controls to limit burst behaviour, repeated retries, and runaway loops.

What's in the full article

Kong's full engineering post covers the operational detail this post intentionally leaves for the source:

• Step-by-step LangGraph code that builds the agent state graph and invokes the OpenAI API
• Examples of OpenAI tool-calling requests and responses that show how the model selects external functions
• A decK declaration that applies Key Auth and AI rate limiting advanced policies at the Kong gateway
• The complete Kong configuration context for running the agent behind an API gateway

👉 Read Kong's engineering post on building and protecting a single LLM agent →

Single LLM agents on Kong AI Gateway: what changes for IAM teams?

Explore further

View Full Forum →  |  NHI Foundation Course →