Tool-calling agents fail in two distinct ways

At a glance

What this is: Arize shows that tool-calling agents break in two different ways, tool choice and tool invocation, and the two failures must be measured separately.

Why it matters: IAM and AI security teams need separate controls for decision quality and action correctness because agentic workflows can fail even when the model appears to pick the right tool.

Context

Tool-calling extends an LLM from text generation into action selection, which creates a governance problem as much as a model-quality problem. A system can appear correct at the conversation level while still choosing the wrong tool or sending an incorrect payload, so evaluation has to separate intent from execution. In agentic workflows, that distinction determines whether the failure is in planning, parameter grounding, or operational control.

For identity and access programmes, this matters because tool use is effectively delegated action. Once an AI system is allowed to invoke tools, teams have to treat the model as a runtime decision-maker whose outputs can trigger side effects, access paths, and downstream automation. The article is therefore less about travel assistance than about how to measure delegated action safely and consistently.

Key questions

Q: How should teams evaluate tool-calling agents when wrong-tool and bad-argument errors both occur?

A: Evaluate tool selection and tool invocation separately. Selection tells you whether the model chose the right capability or correctly chose no tool, while invocation tells you whether the arguments were grounded, complete, and consistent with user intent. A combined score hides where the system fails, so remediation becomes guesswork instead of targeted control design.

Q: Why do tool-calling failures require separate measurement in agentic systems?

A: Because the root causes differ. Wrong-tool errors usually point to planning, routing, or intent recognition problems, while right-tool, bad-argument errors point to grounding, parameter extraction, or context-handling issues. If you do not split them, you cannot tell whether to change the prompt, the evaluator, the schema, or the surrounding tool design.

Q: What are the signs that a tool-calling evaluator is too strict or too loose?

A: Look for disagreements between the judge and the real user intent. A strict evaluator will mark reasonable semantic equivalents as failures, while a loose evaluator will miss invalid arguments, hallucinated values, or unsupported date assumptions. If scores move in the wrong direction after a template change, inspect example-level explanations before you trust the aggregate result.

Q: How should security teams govern AI agents that choose tools at runtime?

A: Security teams should treat runtime tool choice as a governed access event, not a normal application call. That means task-scoped credentials, explicit approval boundaries for sensitive actions, and logs that record both the tool selected and the identity used. If the agent can change its plan, the control model must be able to change with it.

Technical breakdown

Why tool selection and tool invocation are separate failure modes

Tool selection answers whether the model chose the right capability, or correctly chose no tool at all. Tool invocation answers whether the chosen tool received valid, grounded arguments that match user intent. These are not the same problem: a model can select the correct tool and still fail by hallucinating dates, omitting required parameters, or using the wrong value format. If both are rolled into one score, teams cannot tell whether to fix prompting, add constraints, or change the surrounding data model. In agentic systems, separating the two also helps isolate planning errors from execution errors.

Practical implication: Measure tool choice and argument quality as distinct controls so you can fix the right failure mode.

Why reference-free judging helps with tool-calling evaluation

Reference-free LLM-as-a-judge evaluators assess tool use from the conversation and the available tools, rather than relying only on a labeled ground truth. That matters when tool arguments can be semantically equivalent even if they are not string-identical, such as a location written in abbreviated or expanded form. A judge can also evaluate cases where no tool should be called, which string matching cannot do well. The approach is especially useful early in development, when teams want broad coverage before they have perfect labels, but it still requires calibration so the evaluator does not become stricter than the use case needs.

Practical implication: Use reference-free evaluators to cover more cases, then calibrate them against your domain’s acceptable equivalence rules.

Tool-calling evaluation becomes control design, not just scoring

Once tool use reaches production, evaluation is no longer just a benchmark exercise. It becomes part of the control plane around an AI agent, because the system needs checks on whether the model can decide, whether it can invoke, and whether it is operating within acceptable constraints. That aligns closely with agentic AI governance and NHI thinking: the model is not only producing text, it is exercising delegated authority through tools. The real question is whether your controls inspect the decision boundary before action occurs, rather than after the side effect has already happened.

Practical implication: Treat tool-call evaluation as a governance control around delegated action, not a post-hoc model metric.

NHI Mgmt Group analysis

Tool-calling introduces a governance gap between choosing an action and executing it. The article shows that a model can appear accurate while failing in the selection layer or the invocation layer, which means a single success metric is structurally misleading. For agentic AI, that distinction is the difference between intent correctness and action correctness. Practitioners should design evaluation so each layer is visible on its own.

Evaluation quality is now part of the control surface for agentic systems. The article’s examples show that the evaluator itself can be too strict, too loose, or blind to business-specific rules such as date handling. That means teams are not only governing the model, but also governing the judge. This is a reminder that agentic AI assurance needs calibration, not just more scoring.

Delegated tool use creates an identity-and-authorisation problem even when no traditional user login changes. When an LLM can call tools, it is acting through a bounded authority model that resembles machine identity governance more than simple prompt testing. The article makes clear that the important question is not only whether the model is smart, but whether its permitted actions are bounded and measurable. Practitioners should treat tool invocation as authorised runtime behaviour, not just model output.

Separate scoring for selection and invocation is the right named concept here: action-path decomposition. This article shows why collapsing all tool-call errors into one metric obscures where the system actually failed. Action-path decomposition gives teams a clearer way to map planning errors, grounding errors, and tool misuse to different fixes. Practitioners should use this framing when building evaluation pipelines for agents with tool access.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Observability, Audit and Incident Response Guide

What this signals

Action-path decomposition is the practical lesson from tool-calling evaluation: teams need to know whether the failure is in choosing the action or in executing it. If those layers stay merged, model quality, evaluator quality, and workflow quality all blur together, which makes remediation slower and less trustworthy.

In agentic environments, governance shifts from static permission checks to runtime judgment of tool use. That means the evaluation stack has to become part of the control stack, especially when the system can trigger side effects, API calls, or workflow changes on its own.

AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.

For practitioners

• Separate tool selection from tool invocation metrics Track whether the agent chose the right tool independently from whether it passed valid arguments, so prompt fixes do not mask payload failures.
• Calibrate semantic equivalence rules Define when values like city names, airport codes, or date formats should be treated as equivalent in your evaluator, then document that rule in the judge prompt.
• Add environment-specific constraint checks Encode use-case rules such as current-year assumptions, required fields, and allowed value ranges directly into the evaluation template.
• Compare experiments side by side Rerun prompts after each evaluator change and compare the before and after outputs at the example level so you can tell whether you improved the model or just tightened scoring.

Key takeaways

• Tool-calling agents fail in two separable ways, so one aggregate score is not enough to govern them well.
• The article shows that evaluation systems can be wrong in their own right, which means judge calibration is part of the control surface.
• Teams should measure action choice and argument quality independently if they want remediation to be precise and auditable.

Key terms

• Tool Selection Manipulation: An attack in which inputs are crafted to steer an agent toward the wrong tool, interface, or privilege level. The goal is to move a request from a constrained path into a more powerful one, creating unauthorized access to data, systems, or business actions.
• Tool Invocation: Tool invocation is an action where an AI agent calls an external system such as a database, API, or file service. Each invocation should be treated as an auditable identity action because it is the point where the agent can move data, trigger changes, or widen its reach across the environment.
• Reference-Free Evaluation: A scoring approach that judges output without a gold standard answer. It is used for open-ended tasks where the right response is not fixed, including chatbots and agentic workflows. The trade-off is that scoring quality depends on the rubric, the judge model, and the coverage of real-world scenarios.
• Action-Path Decomposition: Action-path decomposition is the practice of separating decision quality from execution quality when evaluating agentic systems. It helps teams locate whether a failure sits in selection, argument grounding, or a downstream control boundary, which makes remediation much more precise.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and agentic AI identity. It gives security and identity practitioners a shared control language for delegated runtime access.