How do organisations decide when to use model routing instead of a single fixed model for agents?

Why This Matters for Security Teams

Model routing is not just a cost optimisation choice. For agents, it is a control decision that affects quality, latency, and the blast radius of bad outputs. A single fixed model can be predictable, but it can also be wasteful for simple tasks and underpowered for high-stakes reasoning. Routing lets security and platform teams match model capability to request risk, which is why governance should define when a request may move to a stronger model, a cheaper model, or a safe fallback.

The real issue is that agentic workflows are dynamic. One tool call may need only classification, while the next step may require code generation, summarisation, or policy reasoning. That variability is why current guidance increasingly treats routing as part of the control plane, not a convenience layer. NHI Mgmt Group notes that 90% of IT leaders say properly managing NHIs is essential for a successful zero-trust implementation in its Ultimate Guide to NHIs, and the same principle applies here: capability should be granted and selected based on context, not habit.

Practitioners should also consider the security implications of model choice. Higher-capability models can increase the chance of unsafe autonomy if prompts, tools, or data boundaries are weak, while cheaper models may miss subtle abuse or policy violations. In practice, many security teams discover routing gaps only after an agent has already made a poor model choice and produced a costly or unsafe action, rather than through intentional policy design.

How It Works in Practice

Organisations usually decide on routing by defining classes of work and the model policy for each class. A common pattern is to route low-risk, high-volume requests to a smaller model, then escalate only when the request crosses an uncertainty, sensitivity, or impact threshold. That threshold can be based on prompt length, required tool access, business impact, data classification, or confidence scoring from the first-pass model. For agentic systems, the decision is often made per step, not per session.

In mature environments, routing logic is expressed as policy, not hardcoded logic. Teams use policy-as-code so that the router can evaluate request context at runtime, alongside workload identity and tool permissions. This aligns well with the NIST AI Risk Management Framework and the OWASP Agentic AI Top 10, both of which push organisations toward measurable risk controls rather than opaque model selection.

• Use a lightweight model for classification, extraction, and trivial summarisation.
• Escalate to a stronger model for multi-step reasoning, code changes, or policy-sensitive actions.
• Apply a fallback path when latency, outage, or quality signals cross defined thresholds.
• Bind routing to workload identity so the agent, tool, and model choice are all attributable.
• Log the reason for each route decision for audit and later tuning.

Routing is most defensible when the organisation can measure quality, cost, and latency separately for each request class. If the router cannot explain why a stronger model was chosen, governance is too loose. The control also benefits from lessons in agent safety research, including NHIMG’s OWASP NHI Top 10 coverage and the CSA MAESTRO agentic AI threat modeling framework, which both emphasize runtime context and chain-of-action risk. These controls tend to break down in highly concurrent, tool-rich workflows because routing decisions happen too late to prevent an unsafe downstream tool invocation.

Common Variations and Edge Cases

Tighter routing often increases operational overhead, requiring organisations to balance consistency against cost and latency. That tradeoff becomes sharper when teams use multiple vendors, regulated datasets, or agents that can chain actions without human review. In those environments, the “best” model may change mid-task, but switching too often can create instability, harder debugging, and inconsistent outputs.

Best practice is evolving for cases where the request is ambiguous or the cost of a wrong choice is high. Some organisations use a fixed model for safety-critical steps and routing only for non-sensitive augmentation. Others use a small model as a gatekeeper that decides whether to escalate. There is no universal standard for this yet, but the selection policy should be explicit about the failure mode it is optimising for: cost, latency, accuracy, or containment.

One important edge case is fallback design. A fallback model should not silently inherit permissions or tool access that the primary model did not have. Another is prompt injection resistance: a routed agent can still be manipulated into requesting a stronger model for the wrong reason, so the router must validate the context, not just the request text. For high-risk workloads, combine routing with real-time policy checks and a tested escalation ceiling. Where model performance is highly volatile, routing may become less useful than a single well-governed model plus stricter tool controls.

Related resources from NHI Mgmt Group

• How do security teams decide when to use custom AI agents instead of fixed workflows for security operations?
• Why do organisations need configurable model routing instead of a single preferred LLM?
• Why do organisations use multiple LLMs instead of a single model for every task?
• Should organisations use the same access model for humans and AI agents?