Topic starter
24/06/2026 9:09 pm
TL;DR: APIs are now the connective tissue for GenAI and agentic workflows, and Kong’s benchmark reports that its AI Gateway outperformed Portkey and LiteLLM on throughput and latency in a controlled EKS test, according to Kong. Performance matters, but identity governance and policy enforcement remain the deciding factors as AI usage moves into production.
Editorial analysis by NHI Mgmt Group, based on content published by Kong: “# AI Gateway Benchmark: Kong AI Gateway, Portkey, and LiteLLM”.
Key questions
Q: Where do AI gateways fail when agent traffic scales quickly?
A: They fail when teams treat them as a transport layer instead of an enforcement point.
Q: Why do agentic workloads change API governance requirements?
A: Agentic workflows chain multiple requests, tools, and model calls together, so governance must track the whole request path rather than one application at a time.
Q: What should teams measure besides gateway latency in AI production?
A: They should measure whether identity context, quota enforcement, and request auditing still work under load.
Practitioner guidance
- Centralise policy enforcement at the AI gateway Apply authentication, token quotas, routing policy, and observability in one request path so model, agent, and tool traffic is governed consistently.
- Classify agent and model consumers explicitly Define which workloads, agents, and downstream services are allowed to transact through the gateway, then bind those decisions to durable identities and scopes.
- Separate performance testing from policy testing Benchmark gateway latency and throughput independently from authorisation, quota, and audit behaviour so load results do not hide governance gaps.
Bottom line: AI gateway design is now an identity and governance issue as much as an infrastructure issue, because agentic traffic depends on the same request path for access and control.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
25/06/2026 6:11 am
API governance is becoming the control surface for agentic AI. When agents, LLMs, and MCP servers all depend on the same traffic path, the gateway becomes the point where identity, policy, and cost controls converge. That makes AI infrastructure governance a cousin of NHI governance, because the practical question is who or what is permitted to invoke downstream services. Practitioners should now assess AI gateways as enforcement infrastructure, not as simple performance middleware.
A question worth separating out:
Q: How should organisations govern LLM, agent, and MCP access together?
A: By using a single policy layer that binds consumer identity, request limits, and observability to every AI interaction path. That approach reduces fragmentation across models, agents, and tools, and it makes revocation and review more practical.
👉 Read our full editorial: AI gateway benchmarks show why API governance now matters for agents