AI gateway routing: what it means for runtime governance teams

TL;DR: An AI gateway can reduce model spend by routing routine work to cheaper models while keeping sensitive tasks on frontier systems, according to JetStream Security, but the real control value is centralized governance over model choice, data path, and tool use. The security case depends on treating routing as runtime policy, not just cost optimisation.

Editorial analysis by NHI Mgmt Group, based on content published by JetStream Security: “How Running an AI Gateway Can Pay for Itself”.

By the numbers:

• OpenRouter’s Claude Code app page showed GLM 5.2 at 811 billion tokens in the 30 days to July 7, sitting right behind Claude Opus 4.8.
• Artificial Analysis put GLM-5.2 at 51.1 on its coding index, compared with 55.7 for Opus 4.8 and 59.9 for Fable 5.
• JetStream Security says a 75/25 split can take a monthly bill from $20,000 to about $8,420.

Key questions

Q: What breaks when an AI gateway is treated as a cost tool instead of a control point?

A: When teams treat the gateway as only a pricing layer, they usually under-govern routing, logging, and exception handling.

Q: Why do AI gateways create governance value beyond model spend reduction?

A: They create value because they concentrate decisions about model choice, data path, and task routing in one place.

Q: What are the signs that an AI routing policy is too weak?

A: The warning signs are uncontrolled fallback to cheaper models, unclear handling of sensitive prompts, tool calls that vary by application instead of policy, and no evidence that routing exceptions are reviewed.

Practitioner guidance

• Define routing policy by workload sensitivity Classify requests into routine, sensitive, and regulated categories, then route each category to an approved model path with explicit escalation rules.
• Separate tool access from model access Keep secrets, shell access, and internet reachability out of the same runtime path wherever possible so model choice does not become tool abuse.
• Measure completed-task cost, not per-token price Compare end-to-end task cost, including retries, caching effects, and tool-call overhead, before you decide which lane is actually cheaper.

Bottom line: AI gateways shift the main control question from which model to buy toward which tasks may flow to which model under which policy.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →