The control plane fragments. Applications start seeing different API shapes, routing rules, budgets, and audit paths depending on which backend serves the request, which makes governance inconsistent and operational troubleshooting harder.
How the control plane fragments
When self-hosted models are added outside the ai gateway, the gateway stops being the single place that defines how requests are handled. Some applications go through the gateway, others call a model endpoint directly, and the result is split policy enforcement, split observability, and split cost control. That is why the same application can behave differently depending on which backend answers.
The practical breakage is not just “more paths.” It is a loss of uniform control over request routing, model selection, quota enforcement, logging, and policy review. Once teams can bypass the gateway, each backend can accumulate its own API shape, its own retry behaviour, and its own audit trail, which makes the platform harder to reason about and govern.
For readers comparing this pattern to the gateway model, the key issue is that the gateway is acting as a control point, not just a proxy. Shadow AI and AI Agent Discovery Guide is relevant here because unmanaged model paths are easiest to spot when you inventory where model calls, OAuth grants, and API keys actually flow.
Why routing, budgets, and audits stop lining up
Once a model is reachable outside the AI gateway, the application logic often starts encoding assumptions about the backend itself. One backend may expect a different request schema, another may support different rate limits, and a third may skip the logging or metadata fields the gateway normally standardises. That creates drift across the stack, even when developers think they are “just adding another model.”
Budgets also become unreliable because spend controls are usually enforced at the shared chokepoint. If a self-hosted endpoint is not in that choke point, you can no longer trust one place to tell you what was called, by whom, and at what cost. Audit paths fragment in the same way, since evidence is now split across gateway logs, application logs, and backend logs that may not share a common request ID or retention policy.
This is the same governance problem that appears in any environment where some requests are centrally mediated and others are not. NIST Cybersecurity Framework 2.0 fits this situation because govern and identify functions depend on knowing which services are in scope and which control point is authoritative.
What teams usually underestimate
The most common mistake is treating self-hosted models as an implementation detail rather than a governance boundary. In practice, the boundary decides which backend is approved, which telemetry is mandatory, which prompts or outputs are retained, and which exceptions need review. If those decisions are left to individual teams, you end up with policy by convention instead of policy by enforcement.
Another subtle break is troubleshooting. When a request fails, the team now has to determine whether the issue is in the app, the gateway, the self-hosted model, or the handoff between them. That increases mean time to diagnose because the failure can look like a model issue, an auth issue, or a routing issue depending on where the request dropped out of the controlled path.
From an identity and access perspective, the important lesson is that model reachability and model authority should not diverge. If a backend can be called outside the gateway, then the bypass path needs equal or stronger authentication, logging, and approval, otherwise the “extra option” becomes an uncontrolled alternate control plane. The LLM Provider API Key Security and LLMjacking Guide is useful here because direct model access often expands credential exposure and makes spend abuse easier to miss.
Risk and Threat Considerations
Adding self-hosted models outside the AI gateway creates a control bypass condition. The main risk is not only inconsistent governance, but also a wider attack and abuse surface, because direct endpoints can inherit weaker routing, weaker logging, and weaker usage limits than the governed path.
Failure mechanism: The gateway no longer enforces a single policy plane, so requests can reach models through alternate paths with different authentication, quota, and audit behaviour. That fragmentation weakens detection and makes unauthorized or excessive usage harder to spot.
Impact: Teams lose assurance over which backend processed a request, which budget it consumed, and which audit trail is authoritative. In the worst case, a bypass path becomes the preferred route for unsafe experimentation, untracked spend, or credential abuse.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0 and NIST SP 800-53 Rev 5 set the technical controls, while ISO/IEC 27001:2022 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.OC-01 — Organizational Context | Multiple model paths change what systems and controls are in scope. |
| GV.RM-01 — Risk Management Strategy | Bypass paths create governance and operational risk that needs explicit treatment. | |
| Recommendation — Define which model backends are governed and keep the authoritative control plane explicit. Classify direct model access as a managed exception with clear risk ownership. | ||
| NIST SP 800-53 Rev 5 | AC-4 — Information Flow Enforcement | A gateway exists to enforce uniform request routing and policy across model paths. |
| AU-2 — Event Logging | Fragmented model access breaks a single authoritative audit trail. | |
| IA-5 — Authenticator Management | Direct model access often relies on separate keys or tokens that need governance. | |
| Recommendation — Enforce model request flows through controlled, policy-bound paths. Log gateway and direct model requests in a consistent, correlated audit stream. Inventory and rotate every credential that can reach a model endpoint directly. | ||
| ISO/IEC 27001:2022 | A.8.15 — Logging | Split backends create split logging and weaken traceability. |
| Recommendation — Centralise logging requirements for every model access path. | ||
Practitioner Guidance
What to verify: Treat every direct-to-model route as a separate control path and verify that it inherits the same request logging, access checks, rate controls, and change approval as the gateway path. If it does not, do not assume the gateway still governs the system.
Decision rule: If a self-hosted model can be called without passing through the gateway, either bring it under the gateway or explicitly document the compensating controls, owner, and audit source for that bypass path. “Temporary” exceptions should be time-boxed, not left as architecture drift.
Practitioner takeaway: The real break is not model diversity, it is losing a single authoritative control plane. The moment routing, budget, and audit decisions depend on backend choice, governance becomes inconsistent and operational risk rises.