How should teams instrument GenAI applications when their framework already emits OpenTelemetry GenAI spans?

Why This Matters for Security Teams

OpenTelemetry GenAI spans are useful only if they preserve enough structure to support investigation, governance, and cost attribution after ingestion. For teams running mixed model, retrieval, and tool-using workloads, the risk is not lack of telemetry, but inconsistent telemetry that cannot be compared across services or environments. The right approach is to treat the emitted gen_ai fields as a control surface, not just an engineering convenience, and to validate that downstream systems keep model identifiers, tool calls, prompt and response metadata, and token usage in a queryable form. That aligns with the NIST Cybersecurity Framework 2.0 emphasis on visibility and monitoring.

Security teams often miss that GenAI instrumentation is also a governance issue. If telemetry is flattened, redacted too aggressively, or mapped inconsistently, incident response loses context and model risk reviews lose evidence. The better question is not whether spans exist, but whether they can support audit, detection, and operational decision-making without bespoke parsing logic. In practice, many security teams encounter telemetry gaps only after a model incident, tool abuse event, or runaway cost spike has already occurred, rather than through intentional validation.

How It Works in Practice

When a framework already emits OpenTelemetry GenAI spans, the goal is to preserve semantic consistency from application to backend. Teams should send spans to a collector or observability pipeline that understands the GenAI semantic conventions, then verify that the receiving platform maps key fields into structured attributes rather than opaque text blobs. That includes model name or version, provider, operation type, prompt and completion tokens, tool invocation details, retrieval metadata, and latency.

A practical implementation usually follows three steps:

• Confirm the application library emits the current GenAI semantic conventions consistently across all services.
• Inspect the collector or exporter to ensure attributes are retained, normalized, and searchable without custom transformation code.
• Use the resulting data for debugging, evaluation, anomaly detection, and chargeback or showback analysis.

For security and governance, the key is to validate what gets exposed to the observability stack. Prompt text, retrieved content, and tool arguments may contain sensitive data, so teams should define field-level retention and redaction rules before broad distribution. NIST guidance on GenAI risk management, including the NIST AI 600-1 GenAI Profile, supports the idea that observability should be usable for risk control, not just performance tracking. That is especially important when spans are used across multiple agents, where one weak link in normalization can break trace continuity. These controls tend to break down when teams mix old and new instrumentations in the same pipeline because attribute names, redaction logic, and trace correlation rules diverge.

Common Variations and Edge Cases

Tighter telemetry governance often increases implementation overhead, requiring organisations to balance observability depth against privacy, storage, and operational cost. There is no universal standard for how much prompt or response detail should be retained yet, so best practice is evolving rather than settled. For regulated environments, the minimum viable approach is usually structured metadata retention with strict controls over sensitive content, while less sensitive development systems may keep richer traces for evaluation and prompt tuning.

Edge cases appear quickly in hybrid estates. If one team uses native OpenTelemetry GenAI spans and another uses custom logs, the observability platform may show partial journeys that are hard to correlate. If agents call external tools or retrieval services, teams should decide whether those dependencies are separate spans, linked spans, or nested operations, and then apply the same convention consistently. That consistency matters for incident review, where missing tool attribution can make a benign retrieval call look like unexplained model behavior. For broader AI governance, the NIST Cybersecurity Framework 2.0 and the NIST AI 600-1 GenAI Profile both point teams toward traceable, reviewable controls rather than ad hoc telemetry. Where this guidance breaks down most often is in high-volume, low-latency inference environments that cannot afford expensive enrichment or synchronous redaction at ingestion.

Related resources from NHI Mgmt Group

• What should teams do when access control spans SAP and other business applications?
• How should security teams respond when a framework RCE affects production applications?
• How should security teams govern GenAI applications without breaking usability?
• Who should own authorization when it spans applications, clusters, and compliance teams?