The rate at which the model picks the correct tool from the search results it receives. This is distinct from retrieval accuracy and is useful for separating model behaviour from search-layer failure in agent tool-routing assessments.
What Selection Accuracy Measures
Selection accuracy measures how often a model chooses the right tool or result from the options it is given, after retrieval has already happened. It isolates the routing decision itself, so teams can tell whether errors come from search quality or from the model’s choice.
Why It Matters in Agent Tool Routing
This metric is most useful when an agent must decide between multiple plausible tools, documents, or actions. High selection accuracy suggests the model can interpret the retrieved set and route work correctly, while low accuracy means the model may be misreading context, overvaluing irrelevant items, or failing to recognise the best match.
Selection accuracy is not the same as end-to-end task success. A system can retrieve the right item and still fail if the model picks the wrong one, and it can also appear weaker than it is if retrieval never surfaces the correct option in the first place.
How It Is Interpreted
Practitioners usually read selection accuracy alongside retrieval accuracy, top-k recall, and downstream task quality. That combination shows whether the weak point is search, ranking, or the final model decision.
The metric is especially important when candidate sets are intentionally broad, such as in tool routing, support triage, or agentic workflows where the model must choose among APIs, actions, or knowledge sources. In those settings, a small drop in selection accuracy can create disproportionate operational error because the wrong choice may still look plausible.
What Good Measurement Looks Like
To be useful, the evaluation set should contain realistic candidate pools and clear ground truth for the correct choice. The candidate set should reflect the kind of ambiguity the model will face in production, not just easy examples where the answer is obvious.
It also helps to separate selection accuracy by task class, tool family, or ambiguity level. That makes it easier to see whether the model struggles with similar tools, long candidate lists, or cases where the right option is semantically close to the distractors.
Risk and Threat Considerations
When selection accuracy is poor, the model may choose an unsafe, expensive, or irrelevant tool even though the correct option was available in the retrieved set. In routed systems, that can create avoidable failure chains because the model’s mistaken choice becomes the next action in the workflow.
Failure mechanism: The model overweights surface similarity, misses subtle tool distinctions, or follows noisy context into the wrong choice, which turns a correct retrieval into an incorrect action.
Impact: The result can be incorrect outputs, broken workflows, accidental data exposure, unnecessary API calls, or downstream agent behaviour that is hard to diagnose because retrieval looked successful.
Practitioner Guidance
What to watch for: Treat selection accuracy as a routing quality metric, not a general intelligence score. If it is low while retrieval quality is strong, focus on prompt design, tool descriptions, candidate ordering, and decision policy rather than on search infrastructure.
Governance implication: For production agent systems, define the correct choice unambiguously and keep the evaluation harness stable over time so changes in selection behaviour are visible. That makes regression testing meaningful when tools, prompts, or model versions change.