Evaluate tool selection and tool invocation separately. Selection tells you whether the model chose the right capability or correctly chose no tool, while invocation tells you whether the arguments were grounded, complete, and consistent with user intent. A combined score hides where the system fails, so remediation becomes guesswork instead of targeted control design.
Separate the evaluation signal before you score the model
When tool-calling agents make both wrong-tool and bad-argument mistakes, the first step is to split the evaluation into two questions: did the model select the right tool, and did it construct a valid call for that tool? Those are different failure modes, and they often have different causes. A single blended score makes it hard to tell whether the issue is routing, grounding, schema adherence, or user-intent interpretation.
A clean separation also helps teams compare runs fairly across prompts, tools, and model versions. If the model chose the wrong tool, no amount of perfect argument formatting fixes the user impact. If the tool choice was correct but the arguments were malformed, the remediation is usually in prompt design, schema constraints, or argument validation rather than capability selection.
Measure the failure at the point it happens
Tool selection is a routing judgment, while tool invocation is an execution judgment. Selection should be scored against capability match or abstention, for example choosing the right function or correctly deciding not to call anything. Invocation should be scored against whether the call was grounded in the user request, complete, type-safe, and consistent with the tool contract.
Practitioners get better diagnostic value when they preserve the intermediate decision. That means recording which tool was proposed, which arguments were emitted, whether validation passed, and whether the call would have produced the intended outcome. This is especially important when one bad selection triggers a bad argument, because the later failure can obscure the earlier root cause.
Where possible, separate “could the agent have solved the task with this tool?” from “did it invoke the chosen tool correctly?” That distinction lets you tune retrieval, planning, schema enforcement, and retry logic independently instead of treating every miss as a generic tool-use failure.
Use separate remediation paths for wrong-tool and bad-argument errors
Wrong-tool errors usually point to capability confusion, weak tool descriptions, ambiguous routing, or poor preference ordering among tools. Bad-argument errors usually point to missing context, weak slot filling, schema mismatch, or the model over-interpreting user intent. Those problems may co-occur, but they do not deserve the same fix.
In practice, teams should build separate dashboards or confusion matrices for tool choice and argument quality. That makes it easier to see whether one tool is being over-selected, whether a specific parameter is repeatedly omitted, or whether the model is selecting the right tool but hallucinating values. If the two failures are merged, regressions can look smaller than they really are, and you end up optimising the wrong layer.
For agentic systems, this separation also improves safety. A bad tool choice can be a policy issue, while a bad argument can become an unsafe action even when the tool choice was correct. Evaluating them independently gives reviewers a clearer basis for deciding whether the fix belongs in policy, schema, prompts, or runtime controls.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 addresses the attack and risk surface, while NIST SP 800-53 Rev 5 sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI02 — Tool Misuse | Tool choice and invocation are distinct agentic tool-use failure modes. |
| ASI03 — Identity & Privilege Abuse | Wrong-tool and bad-argument errors can create unsafe agent actions through misplaced authority. | |
| Recommendation — Score tool selection separately from argument validity and fix each failure path independently. Constrain agent actions per tool and verify request-to-privilege alignment before execution. | ||
| NIST SP 800-53 Rev 5 | AU-2 — Event Logging | Separate telemetry is needed to diagnose routing versus invocation failures. |
| SI-10 — Information Input Validation | Bad-argument errors are fundamentally input and parameter validation problems. | |
| AC-6 — Least Privilege | Incorrect tool calls become more dangerous when the agent has excessive authority. | |
| Recommendation — Log proposed tool choice, arguments, validation results, and outcome as distinct events. Validate tool arguments against strict schemas before the tool is allowed to run. Limit each tool to the minimum permissions needed for its intended action. | ||
Practitioner Guidance
What to verify: Keep the selected tool, the emitted arguments, the validation result, and the final outcome as separate evaluation fields. If you only keep an end-to-end pass/fail score, you lose the ability to tell whether the model failed to reason about the task or failed to execute the call.
Decision rule: If the tool was wrong, prioritise routing and capability selection fixes; if the tool was right but the arguments were wrong, prioritise schema grounding and input-completion controls. Treat combined failures as two defects, not one blended error.
Common mistake: Teams often tune the model against aggregate tool-use accuracy and then wonder why the same “improvement” increases silent misfires. That usually means one failure mode improved while the other deteriorated.
Practitioner takeaway: The most useful evaluation is the one that tells you which layer failed first, because that is what makes remediation targeted instead of speculative.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Free weekly newsletter
Subscribe to the NHI & AI Identity Journal
The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.
Bonus 33% off our NHI Course when you subscribe.
Reviewed and updated by the NHIMG editorial team on October 12, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org