Span-level telemetry records the smallest meaningful unit of work inside an AI run, such as one model call or tool invocation. It lets teams trace cost, retries, and behaviour back to the exact step that created them instead of relying on aggregate totals.
Expanded Definition
Span-level telemetry is the practice of instrumenting each discrete span in an AI workflow so that cost, latency, retries, tool use, and output characteristics can be attributed to the exact step that produced them. In agentic systems, a span may represent a model call, a retrieval action, a guardrail check, a tool invocation, or a handoff between components. This makes the term broader than simple logging, because the span itself becomes the unit of analysis rather than the whole request.
Definitions vary across vendors on where a span begins and ends, especially in systems that combine orchestration, RAG, and autonomous tool use. NHI Management Group treats the useful boundary as the smallest unit that can be independently observed, governed, and investigated without losing causal context. That distinction matters when teams need to reconcile performance, security, and cost telemetry from the same execution path. For a governance baseline, the NIST Cybersecurity Framework 2.0 is relevant because it emphasizes visibility, detection, and response across the full lifecycle of a system.
The most common misapplication is treating aggregate request logs as span-level telemetry, which occurs when organisations cannot isolate which model call, tool action, or retry actually caused the observed issue.
Examples and Use Cases
Implementing span-level telemetry rigorously often introduces instrumentation overhead and data-management complexity, requiring organisations to weigh diagnostic precision against storage, privacy, and operational cost.
- Tracing a multi-step AI agent that searches a knowledge base, calls an LLM, and submits a ticket, so each span shows where latency or failure was introduced.
- Separating model inference cost from retrieval and tool execution cost in order to identify which step drives spend during high-volume usage.
- Capturing retry behaviour when a downstream API fails, allowing teams to see whether the model, the orchestration layer, or the tool integration triggered repeated execution.
- Recording span attributes for policy enforcement, such as whether a guardrail blocked a prompt, whether a tool was approved, or whether a secret was accessed during execution.
- Correlating telemetry with observability standards such as OpenTelemetry traces to preserve context across distributed AI components.
In practice, span-level telemetry is most useful when teams need to compare one agent path against another, or when a single workflow contains several model and tool interactions that produce different security and cost outcomes. It is also valuable for incident review when a prompt, retrieval result, or external action must be tied to a specific execution step.
Why It Matters for Security Teams
Security teams need span-level telemetry because AI incidents rarely originate in a single monolithic event. They emerge from a sequence of decisions, tool calls, and external interactions that only become visible when the workflow is decomposed into spans. That makes the concept especially important for agentic AI, where an autonomous software entity with execution authority may trigger side effects across multiple systems. Without span-level visibility, teams cannot reliably determine whether a failure was caused by prompt manipulation, unsafe tool selection, excessive retries, or an unexpected dependency response.
This matters for identity and secrets governance as well. If a span shows that an AI agent accessed a token, called an internal service, or attempted a privileged action, the organisation can trace whether the control failed at authorisation, orchestration, or execution time. Span-level telemetry therefore supports investigation, auditability, and least-privilege enforcement across AI operations. For control mapping and security program alignment, teams can also anchor telemetry requirements to the NIST Cybersecurity Framework 2.0 and related tracing practices in distributed systems.
Organisations typically encounter the need for span-level telemetry only after an AI workflow misbehaves in production, at which point the lack of step-level evidence makes containment and root-cause analysis operationally unavoidable.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and CSA MAESTRO address the attack and risk surface, while NIST CSF 2.0, NIST AI RMF and NIST AI 600-1 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | DE.AE-1 | Telemetry supports anomaly detection by exposing suspicious activity at span granularity. |
| NIST AI RMF | AIRMF stresses observability and traceability for trustworthy AI lifecycle governance. | |
| NIST AI 600-1 | The GenAI profile emphasizes logging and monitoring of generative AI behaviors and outputs. | |
| OWASP Agentic AI Top 10 | Agentic AI guidance depends on tracing actions, tool use, and escalation paths. | |
| CSA MAESTRO | MAESTRO addresses observability for agentic workflows and their control points. |
Design span telemetry to support traceability, monitoring, and incident investigation across AI systems.
