AI connectivity governance is becoming the control layer for agents

At a glance

What this is: This is an analysis of AI connectivity as a control layer for securing, routing, observing, and governing enterprise AI interactions, with a focus on why uncontrolled AI traffic and agent activity create operational and identity risk.

Why it matters: It matters because IAM, NHI, and governance teams need one policy layer for model calls, agent actions, and prompt/data flows before shadow AI and unmanaged automation outgrow existing controls.

👉 Read Kong's analysis of AI connectivity governance for enterprise AI traffic

Context

AI connectivity is the governance layer that sits between applications, models, and the data they exchange. In practice, it becomes relevant when AI usage stops being a series of isolated pilots and starts behaving like shared enterprise infrastructure with identity, policy, and observability requirements.

That shift matters for IAM and NHI teams because AI systems now combine authentication, routing, content handling, and audit logging in ways that traditional API management did not need to address. The core challenge is not just traffic volume, but the fact that AI interactions can be initiated by agents, carry sensitive context, and span multiple providers and control planes.

Kong frames this as the move from experimentation to operational chaos, and that is a fair description of what happens when AI connectivity is left decentralised. The starting position is increasingly typical, not exceptional.

Key questions

Q: How should security teams govern AI connectivity across multiple models and providers?

A: Security teams should govern AI connectivity with a central policy layer that handles authentication, authorisation, logging, redaction, and quota enforcement across all providers. The key is consistency. If every team implements its own controls, auditability breaks down and AI traffic becomes impossible to govern at enterprise scale.

Q: Why do AI agents create governance gaps for IAM and NHI teams?

A: AI agents create governance gaps because they can initiate actions, route context, and chain API calls in ways that conventional IAM models do not expect. Existing controls often assume a stable human or service account path. When the actor can vary its path at runtime, visibility, accountability, and policy enforcement all become harder.

Q: What breaks when shadow AI is not visible to the security team?

A: When shadow AI is invisible, security teams lose control over where prompts go, what data is exposed, and which identity is consuming AI services. That creates compliance risk, budget risk, and data leakage risk at the same time. Discovery is the prerequisite for any credible governance model.

Q: Which frameworks help teams structure AI connectivity governance?

A: Teams should align AI connectivity governance with Zero Trust and identity lifecycle discipline, then extend policy to data handling and auditability. For agentic use cases, the governance model should also reflect AI risk management and agent-specific threat modelling so that access, context, and actions are managed together.

Technical breakdown

Why AI connectivity differs from traditional API connectivity

Traditional API traffic is deterministic: a known request typically produces a predictable response, and policy can be enforced around fixed endpoints and stable schemas. AI connectivity is different because model calls are context-aware, non-deterministic, and often semantically routed. The same user intent may be satisfied by different providers, memory stores, or tool chains depending on token cost, latency, policy, and context. That means the control layer has to understand not just who is calling, but what data is being sent, where context is sourced, and which downstream model or tool is allowed to act. In AI systems, observability and authorisation have to extend beyond the API edge into the interaction itself.

Practical implication: treat AI traffic as a governed identity and data path, not as ordinary API traffic with a different payload.

How semantic caching and token governance change the operating model

Semantic caching reduces repeated model calls by storing responses based on meaning rather than exact text. That lowers latency and cost, but it also shifts governance from request counting to intent-sensitive policy enforcement. Token-aware limits matter because AI cost is usage-shaped, not session-shaped, and one runaway workflow can consume budget rapidly if quotas and caching are absent. From an identity perspective, this is another reason AI connectivity needs central control: the same actor may generate many distinct prompts, each with different data sensitivity and different cost impact. The governance model must therefore tie policy to the interaction context, not just to the application or user account.

Practical implication: set quotas, caching rules, and usage controls at the gateway layer so cost and policy stay aligned.

What AI gateways change for authentication, redaction, and auditability

An AI gateway is not just a traffic router. In mature deployments it becomes the enforcement point for authentication, authorisation, prompt filtering, PII redaction, and audit logging across models and teams. That matters because AI prompts can carry secrets, personal data, and internal context into external services, while autonomous workflows can chain multiple calls without human review between steps. Gateway-level controls create consistency across providers and reduce the need to reimplement security in every application. They also make audit trails usable, because the same policy engine can record tokens, models, request context, and enforcement outcomes in one place.

Practical implication: centralise authentication, redaction, and logging in the AI control plane instead of distributing them across every app team.

NHI Mgmt Group analysis

AI connectivity is becoming the policy boundary for machine identity at scale. Once AI services, agents, and application workflows share the same interaction layer, the old assumption that identity is provisioned once and then merely used no longer holds. Every model call, tool invocation, and context fetch becomes part of the access model. The implication is that NHI governance now has to cover AI traffic patterns as a first-class runtime concern, not a side effect of application integration.

Shadow AI is really an identity discovery problem. Unmanaged AI usage is not just an unsanctioned tooling issue, it is an unobserved access pattern problem. When teams can launch model calls directly, route around central policy, or leak prompts into public services, security loses the ability to establish who or what accessed which context. That breaks visibility, audit, and accountability at the same time, and it is why governance must start with discovery before policy can be credible.

AI gateways compress three controls into one architectural decision: access, cost, and data handling. In older stack models those controls lived in separate tooling, which created drift between security, finance, and operations. AI connectivity unifies them because the same exchange can consume tokens, expose sensitive data, and trigger downstream actions. Practitioners should treat that compression as a design requirement, not an optimisation, because isolated control planes will miss the interaction-level risk.

Runtime policy for AI is now a lifecycle issue, not just a deployment issue. As agentic workflows expand, the important question is no longer whether the model is reachable but whether the access path can be governed across creation, change, and retirement. That puts AI connectivity into the same governance conversation as workload identity and access review, even when the workload is a model-mediated process rather than a conventional service account. The practical conclusion is that lifecycle discipline has to move into the AI path itself.

Context engineering is becoming an identity problem because context determines privilege in practice. When a model or agent is fed the wrong data at the wrong time, it can infer or act beyond its intended scope even if the original account permissions looked reasonable. That makes data routing, model selection, and memory access part of the entitlement story. Practitioners should treat context distribution as a governance surface, not just as an application design choice.

From our research:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
• Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
• For a wider view of the control problem, see OWASP Agentic AI Top 10 for the runtime risks that emerge when agents can select tools and act on context.

What this signals

AI connectivity will become the enforcement point for agent governance. As AI usage spreads across teams, security programmes will need one place to measure model access, prompt handling, and tool invocation. The organisations that can route AI traffic through a governed control plane will be able to prove who accessed what, while everyone else will be left reconstructing behaviour after the fact.

With 80% of organisations already seeing agents act beyond intended scope, the governance problem is no longer theoretical. That figure points to a structural mismatch between how enterprises deploy AI and how they currently assign responsibility for identity, data, and runtime action. Practitioners should expect more pressure to integrate discovery, policy enforcement, and audit evidence into the same operating model.

Context is becoming the new entitlement surface. When prompts, memory, and retrieved documents determine what an AI system can do in practice, policy must extend beyond login and token issuance. Teams should prepare to manage context flow with the same discipline they already apply to secrets, privilege, and data classification.

For practitioners

• Inventory AI entry points and model routes Map every application, agent, and workflow that can call an LLM or external AI service, then record which provider, tool chain, and data class each path can reach. This should include shadow AI usage discovered outside approved platforms.
• Centralise policy at the AI gateway Enforce authentication, authorisation, redaction, and audit logging in one control plane so teams do not implement inconsistent safeguards in application code. Use the gateway to apply consistent rules across providers, not per-team exceptions.
• Bind token controls to business ownership Set quotas, rate limits, and cost attribution by application, team, and use case so runaway usage can be traced quickly. Token governance should be reviewed alongside access reviews because spend and access often drift together.
• Classify prompts and context before they reach models Apply PII detection, redaction, and content policy checks before prompts leave the trust boundary. Treat retrieved context, memory, and tool output as governed data sources rather than neutral inputs.

Key takeaways

• AI connectivity turns model access, data handling, and cost control into one governance problem that IAM and NHI teams can no longer separate.
• AI agents and shadow AI increase operational risk because organisations lose visibility into what was accessed, where it went, and what it cost.
• Central policy enforcement at the gateway layer is the practical way to keep authentication, redaction, logging, and quotas aligned across AI usage.

Key terms

• AI Connectivity: AI connectivity is the enterprise control layer for governing how applications, users, and agents reach AI services. It combines routing, security, observability, and policy enforcement so AI interactions can be managed consistently across providers and use cases.
• AI Gateway: An AI gateway is the enforcement point that sits between applications and AI services. It applies authentication, authorisation, redaction, logging, and usage controls before and after model calls, giving security and platform teams one place to govern AI traffic.
• Shadow AI: Shadow AI is AI usage that exists outside approved governance and monitoring. It includes unsanctioned model access, untracked prompts, and private workflows that bypass enterprise policy, creating blind spots for security, compliance, and data protection.
• Semantic Caching: Semantic caching stores AI responses by meaning rather than exact wording. It reduces repeated model calls, lowers latency, and cuts token spend, but it also requires governance so cached content does not bypass policy, classification, or data handling rules.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.

This post draws on content published by Kong: Managing the Chaos: How AI Gateways Enable Scalable AI Connectivity. Read the original.