The process of sending a prompt from one platform to another underlying model provider for inference. This matters because a user may trust the front-end service while the actual content is processed elsewhere. Routing affects confidentiality, identity exposure, and which organisation ultimately handles the data.
Expanded Definition
Hosted model routing describes a service pattern where a front-end application receives a prompt, then forwards that content to a separate model host or infrastructure provider for inference. The user experience may appear single-vendor, but the processing path can involve multiple organisations, contracts, and security boundaries. In practice, this means the service that collects the input is not always the same entity that executes the model call, stores telemetry, or applies downstream moderation.
For security teams, the key issue is not routing itself but the accountability and data-flow implications created by the handoff. That includes where prompts are retained, which entity can observe secrets or personal data embedded in the prompt, and whether the route changes by geography, tenancy, or model availability. This is closely related to governance concerns in NIST Cybersecurity Framework 2.0, especially where third-party risk, asset visibility, and data handling are involved.
Usage in the industry is still evolving, and definitions vary across vendors when hosted routing is bundled with proxying, orchestration, or model brokerage. The most common misapplication is assuming the front-end vendor is the sole processor of the data, which occurs when organisations fail to trace the actual inference destination and the logs or caches associated with it.
Examples and Use Cases
Implementing hosted model routing rigorously often introduces visibility and contractual overhead, requiring organisations to weigh convenience and model flexibility against tighter data-governance controls.
- A customer support portal sends questions to one branded AI service, but the prompt is routed to a separate foundation model provider for response generation.
- An internal copiloting tool routes prompts between different model hosts depending on latency, region, or cost, creating multiple inference paths that must be documented.
- A regulated enterprise uses hosted routing to keep a familiar interface while moving sensitive workloads to an approved provider under a distinct processing agreement.
- A security team reviews whether routed prompts containing secrets, tokens, or personal data are filtered before leaving the initial platform and whether the destination retains them.
- A procurement team validates whether the routing layer changes the effective data controller, processor, or subprocessor chain for obligations under NIST CSF-aligned governance.
The strongest implementations document the routing path as a first-class dependency, not an implementation detail. That means inventorying each upstream and downstream service, checking what telemetry is captured, and confirming whether the same prompt can be sent to different hosts over time.
Why It Matters for Security Teams
Hosted model routing matters because it can quietly change the trust boundary around AI use without changing the user interface. Security teams may think they are protecting one application, when in reality they are governing a chain of providers with different retention rules, authentication models, and support access. That creates exposure for confidential content, customer data, and embedded credentials if prompts are not sanitised before forwarding.
This is also an identity issue. When routing is mediated by API keys, service accounts, or delegated tokens, the organisation must know which identity is authorised to send data, which identity is accepted by the downstream model host, and whether those credentials are reusable across environments. For this reason, hosted routing overlaps with third-party risk, secrets management, and agentic AI governance, especially where autonomous tools can trigger model calls without direct human review. A useful governance anchor is again the NIST Cybersecurity Framework 2.0, which helps teams structure visibility, protection, and oversight around external dependencies.
Organisations typically encounter the real risk only after a data leak, unexpected retention discovery, or vendor investigation reveals that prompts were routed to a different processor than the one originally approved, at which point hosted model routing becomes operationally unavoidable to address.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 address the attack and risk surface, while NIST CSF 2.0, NIST AI RMF and NIST SP 800-63 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.SC-1 | Hosted routing creates third-party and supply-chain governance obligations across model providers. |
| NIST AI RMF | GOVERN | AI RMF governance applies to accountability, transparency, and risk management in routed AI services. |
| NIST SP 800-63 | Identity assurance matters when service identities or delegated credentials invoke downstream model hosts. | |
| OWASP Agentic AI Top 10 | Agentic systems may route prompts and tool calls through hidden providers or intermediaries. |
Inventory routed model dependencies and assign oversight for each provider in the service chain.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on August 21, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
