Prompt-token tracking tells you how much input a call consumed, while span-level tracking shows which step in the workflow burned the tokens. The first is useful for prompt design and context sizing. The second is essential for agent traces, where retries, tool calls, and sub-agent loops can dominate cost.
Why these two tracking models answer different operational questions
Prompt-token tracking and span-level token tracking both measure usage, but they answer different management questions. Prompt-token tracking is a coarse cost and sizing signal for an individual call. Span-level tracking is a diagnostic signal for a workflow, showing where consumption occurred across the trace, including retries, nested steps, and tool-heavy paths.
The practical difference is granularity. If you only need to estimate prompt size, tune context windows, or watch per-request cost, prompt-token tracking is usually enough. If the system is an agentic workflow, the number that matters is often not the whole request, but the step that repeated, delegated, or expanded the trace.
That is why span-level visibility is especially useful when a single user action fans out into multiple model calls. It lets teams separate the prompt that started the run from the tool invocation, retry, planner step, or sub-agent loop that actually consumed the budget.
What prompt-token tracking can and cannot tell you
Prompt-token tracking is best understood as input accounting. It tells you how much context went into a call, which helps with prompt engineering, truncation risk, and estimating whether a request is likely to stay within model limits. It is the right lens when the main question is, “How expensive or large was this prompt?”
What it does not show is where the spend came from inside a larger workflow. A call may look efficient at the prompt level while the surrounding process quietly multiplies cost through retries, tool calls, or repeated planner evaluations. In that situation, prompt-level numbers can look healthy even when the trace is inefficient.
For cost control, that means prompt-token tracking is a front-end metric, not a full execution metric. It is useful for individual prompt design, but it will not explain why one run cost five times another if the difference happened downstream in the trace.
Why span-level tracking is the better lens for agents and traces
Span-level token tracking ties token consumption to the step that produced it. That makes it the better choice for agent traces, orchestration layers, and any workflow where one logical task can trigger multiple model invocations. It shows which span, operation, or sub-step is responsible for the spend, rather than hiding everything inside one aggregate request total.
This matters because agent costs are often dominated by control flow, not by the original prompt. A planner may re-query, a tool may trigger follow-up summarisation, or a sub-agent may loop until it finds a usable result. Without span-level telemetry, those patterns are hard to attribute and even harder to optimise.
Span-level tracking also improves governance of complex systems. It helps teams distinguish between expected multi-step behaviour and accidental inefficiency, such as unnecessary retries, duplicated retrieval, or a tool chain that reuses context too aggressively.
Risk and Threat Considerations
When token accounting stays at the prompt level, teams can underestimate real consumption and miss runaway behaviour in agent workflows. That creates operational risk, especially where retries, tool chains, or recursive loops can turn a small request into a large bill or a degraded service experience.
Failure mechanism: Aggregate prompt metrics hide the expensive span, so repeated calls, nested tools, or sub-agent loops look like normal usage until the workload crosses budget, latency, or quota thresholds.
Impact: Teams lose the ability to attribute cost to the exact workflow step, making it harder to control spend, diagnose regressions, or contain runaway agent behaviour before it affects users or budgets.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 addresses the attack and risk surface, while NIST SP 800-53 Rev 5 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | AU-6 — Audit Record Review, Analysis, and Reporting | Token-by-span telemetry supports trace analysis and attribution of costly workflow steps. |
| AU-12 — Audit Record Generation | Span-level tracking depends on generating detailed execution records for each step. | |
| CM-7 — Least Functionality | Granular token visibility helps identify unnecessary tool calls and repeated execution paths. | |
| Recommendation — Instrument spans and review traces to attribute repeated token burn to the exact workflow step. Generate step-level audit records so token usage can be traced to individual spans. Remove or constrain workflow steps that create avoidable token consumption. | ||
| NIST CSF 2.0 | GV.OV-01 — Oversight of cybersecurity risk management strategy | Comparing prompt and span telemetry improves oversight of workflow cost and control effectiveness. |
| Recommendation — Use trace-level metrics to oversee whether AI workflows stay within expected cost and behaviour. | ||
| OWASP Agentic AI Top 10 | ASI08 — Cascading Failures | Retries and sub-agent loops can amplify token use across chained workflow steps. |
| Recommendation — Detect and bound repeated agent steps before they cascade into excessive cost. | ||
Practitioner Guidance
What to prioritise: Use prompt-token tracking for prompt design and context sizing, but make span-level tracking the default for agentic workflows, orchestration layers, and any system with retries or tool calls. If you cannot attribute token burn to a span, you cannot reliably optimise the workflow.
What to verify: Confirm that your tracing pipeline preserves step boundaries across planner calls, tool execution, retries, and sub-agents. A useful test is whether you can identify the single span that caused the largest incremental token increase in a run.
Common mistake: Treating one per-request token count as if it explains the whole workflow. That is usually enough for a single prompt, but it is misleading when the real cost comes from execution paths inside the trace.
Practitioner takeaway: Prompt-token tracking tells you what the call consumed; span-level tracking tells you which part of the system caused it, and that distinction is what makes agent cost control actionable.
Related resources from NHI Mgmt Group
- What is the difference between managed identities and hardcoded secrets for AI agents?
- What is the difference between human identity governance and AI agent governance?
- What is the difference between workload identity and API keys for AI agents?
- What is the difference between governing human access and governing AI agent access?
Deepen Your Knowledge
Free weekly newsletter
Subscribe to the NHI & AI Identity Journal
The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.
Bonus 33% off our NHI Course when you subscribe.
Reviewed and updated by the NHIMG editorial team on October 12, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org