What are the signs that span-level evaluation is not enough for an LLM workflow?

Span-level evaluation is not enough when each individual step appears correct, but the final answer still misses the user’s goal. Common signs include the right tools being called in the wrong order, valid intermediate outputs that do not combine into a relevant result, or a workflow that is technically successful but operationally useless.

Why span-level checks miss workflow failure

Span-level evaluation is useful for catching local mistakes, but it can miss the larger question of whether the workflow actually solved the user’s task. A sequence of individually plausible steps can still fail if the system loses intent, chooses a valid but irrelevant tool path, or produces an answer that is coherent in isolation but wrong in context. For workflow owners, that distinction matters because local quality does not guarantee task completion. The NIST AI Risk Management Framework is relevant here because it frames AI systems around trustworthy outcomes, not just isolated component behaviour. In practice, many teams discover the gap only after users report that the system “looked right” at each step but still failed to deliver the needed result.

How workflow-level failure shows up in practice

The clearest sign is a mismatch between intermediate correctness and end-to-end usefulness. The model may select the right retrieval source, call the right tool, or produce a valid sub-answer, yet the assembled result still ignores the user’s intent, misses a constraint, or answers a different question. That usually means the evaluation surface is too narrow. Span checks can confirm formatting, syntax, or local factuality, but they rarely show whether the workflow preserved task context across turns, branches, or tool calls.

Common workflow-level symptoms include:

Steps are technically valid, but the final answer is incomplete or off-target.
Tool use succeeds, yet the orchestration order prevents the system from building the right context.
Intermediate outputs are correct in isolation but incompatible when combined.
Recovery paths, retries, or fallback logic change the outcome in ways span checks do not measure.

That is why workflow evaluation needs a task-level success criterion, not just a span-level one. For agent-heavy systems, the OWASP Top 10 for Agentic Applications 2026 is a useful external lens because it highlights how orchestration and agent behaviour can fail even when individual actions appear safe. Span-level evaluation breaks down when the system’s real failure happens at the boundary between steps, where no single span looks obviously wrong.

Where the simple evaluation model breaks down

Stricter span checks often increase review overhead, requiring teams to balance local precision against the cost of missing end-to-end failures.

One edge case is a workflow that is reliable on straightforward prompts but fails on multi-step requests with dependencies, ambiguity, or conditional logic. Another is a system that performs well under automated test prompts yet degrades when a user’s actual intent requires synthesis across multiple retrieved facts. There is also a governance issue: teams may treat a clean span as evidence of overall quality, when in reality it only proves that one segment was acceptable.

Guidance varies across organisations, but there is broad consensus that evaluation must match the system boundary being trusted. If the production promise is a completed task, then the evaluation target must be task completion, not just local correctness. That is especially true when tool calls, branching logic, or memory influence the final result.

If a workflow is composed of individually correct spans but still fails user intent, the evaluation design is too granular for the decision being made. The fix is not more confidence in each span; it is a wider success criterion that measures whether the whole workflow produced the right outcome.

Risk and Threat Considerations

When span-level evaluation is treated as sufficient, the main risk is false confidence: teams may approve workflows that are locally correct but operationally unsafe, unreliable, or easy to game. In LLM systems, that can create blind spots around prompt injection, tool misuse, broken orchestration, and outcome drift that only appear at the task level.

Failure mechanism: The weakness emerges when evaluation is anchored to intermediate spans instead of the full decision chain. An attacker, malformed input, or ordinary edge case can steer one step without tripping span checks, then exploit the gap between correct sub-outputs and incorrect final assembly.

Impact: The workflow may return misleading answers, take the wrong action, leak trust into a bad tool path, or appear validated despite failing the user’s real objective.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

OWASP Agentic AI Top 10 and MITRE ATLAS address the attack and risk surface, while NIST AI RMF, NIST AI 600-1 and CIS Controls v8 set the governance and control requirements practitioners need to meet.

Framework	Control / Reference	Relevance
NIST AI RMF	GV-1 — Govern	Workflow evaluation must align with trustworthy AI outcomes and oversight.
Recommendation — Define task-level success criteria that reflect the AI system’s intended outcome.
NIST AI 600-1	M.2 — Measure Validity and Reliability	Span checks can miss whether the full workflow remains reliable end to end.
Recommendation — Measure end-to-end task success, not only isolated step quality.
OWASP Agentic AI Top 10	A2 — Insecure Tool Usage	Agent workflows can look correct per step while failing through bad orchestration or tool selection.
Recommendation — Audit tool-call sequences and verify that orchestration preserves the user’s intent.
MITRE ATLAS	AML.T0038 — Model Output Manipulation	Local correctness can mask adversarial steering that changes the final workflow outcome.
Recommendation — Hunt for input patterns that alter downstream outputs without breaking individual spans.
CIS Controls v8	8.2 — Audit Log Management	End-to-end traces are needed to detect workflow failures that span checks alone miss.
Recommendation — Retain full workflow traces so reviewers can reconstruct how the final outcome was produced.

Practitioner Guidance

What to prioritise: Evaluate the unit of trust you actually ship. If the product promise is a completed task, define success at the workflow level, not at the span level, and include end-state correctness in your acceptance criteria.

What to verify: Check whether a “passing” span still produces failure when combined with neighbouring steps, retries, or tool outputs. If the answer can be locally sound and globally wrong, span-only review is incomplete.

What practitioners underestimate: The most common mistake is assuming that clean intermediate traces prove reliability. They do not. A workflow can be internally well-formed and still fail the user because the system preserved correctness without preserving intent.

Practitioner takeaway: Treat span evaluation as a diagnostic lens, not a release gate, whenever orchestration, tool use, or multi-step reasoning determines whether the system actually solves the task.