Tool-calling agents: are your evaluation controls keeping up?

TL;DR: Tool-calling LLMs fail in two distinct ways, either choosing the wrong tool or choosing the right tool with bad arguments, and Arize shows these errors need separate measurement because one hides selection problems while the other exposes invocation quality. That distinction matters because agentic systems create governance and reliability risks that conventional evaluation workflows often blur.

NHIMG editorial — based on content published by Arize: How to Evaluate Tool-Calling Agents

Questions worth separating out

Q: What breaks when an AI agent can call tools but tool selection and invocation are not measured separately?

A: You lose visibility into which failure actually occurred.

Q: Why do tool-calling agents create governance problems for security and IAM teams?

A: Because the model is no longer only producing text.

Q: How do teams know whether an agent evaluation judge is too strict or too loose?

A: Look for disagreement cases where the model’s output is operationally defensible even if it does not match the reference exactly.

Practitioner guidance

• Implement separate tool-selection and tool-invocation scores Track whether the model picked the correct tool and whether it passed valid arguments as different control outcomes.
• Calibrate judges against semantic equivalence Review cases where different values are still operationally valid, such as location names or equivalent entity names.
• Add domain-specific constraints to evaluator prompts Encode rules that matter in production, such as current-year date handling, required parameter formats, and disallowed hallucinated values.

What's in the full article

Arize's full tutorial covers the operational detail this post intentionally leaves for the source:

• The exact Phoenix evaluator templates for tool selection and tool invocation, including how the judge input is structured.
• The notebook workflow for loading the travel assistant dataset, running experiments, and comparing versions in the UI.
• The prompt edits used to fix year handling and calibrate semantic equivalence for location arguments.
• The custom evaluator pattern for matching tool calls against labeled ground truth without relying on string equality.

👉 Read Arize's tutorial on evaluating tool-calling agents →

Tool-calling agents: are your evaluation controls keeping up?

Explore further

View Full Forum →  |  NHI Foundation Course →