Notifications
Clear all
Topic starter
21/08/2026 4:43 pm
TL;DR: AI observability and governance break when teams cannot reconcile tokens, model names, inputs, outputs, and span types across conventions, according to Fiddler. The practical issue is not collection but semantic consistency: without it, platform teams cannot compare, alert, or enforce reliably across AI stacks.
NHIMG editorial — based on content published by Fiddler: Server-Side Semantic Mapping for Consistent AI Observability Across Frameworks
By the numbers:
- Fiddler documents more than 30 semantic concepts across token usage, cost, model and provider identity, agents and tools, sessions, content, performance, span identity, and metadata.
- It also provides more than 140 predefined mappings across roughly a dozen frameworks and SDKs.
Questions worth separating out
Q: How should security teams normalise AI telemetry across multiple frameworks?
A: Use a central semantic mapping layer that translates framework-specific attribute names into canonical concepts after ingestion.
Q: Why does OpenTelemetry not fully solve AI observability governance?
A: OpenTelemetry standardises transport, not meaning.
Q: What breaks when AI telemetry semantics are inconsistent?
A: Cross-framework comparison breaks first, followed by alert quality, usage reporting, and evaluator consistency.
Practitioner guidance
- Define canonical AI telemetry concepts Create a controlled vocabulary for core AI observability fields such as input tokens, output tokens, model identity, tool execution, and agent spans.
- Preserve raw and mapped telemetry side by side Store the original attribute key together with the resolved semantic concept so engineers can verify mappings and investigate anomalies.
- Treat runtime mapping changes as governed configuration Allow mappings to be updated without redeploying applications, but place those changes under change control and review.
What's in the full article
Fiddler's full blog post covers the implementation detail this post intentionally leaves at the architectural level:
- The complete list of semantic concepts used to normalise AI telemetry across frameworks and SDKs.
- Examples of how raw attributes are preserved alongside mapped concepts for debugging and verification.
- Runtime mapping workflows for adding or deleting mappings without changing application code.
- The broader control-plane context that links semantic mapping to monitoring, enforcement, and governance.
👉 Read Fiddler's deep dive on server-side semantic mapping for AI observability →
AI telemetry semantics across frameworks: are your controls keeping up?
Explore further
21/08/2026 5:03 pm
Semantic drift is becoming an AI governance problem, not just an observability problem. When the same AI event is labelled differently across SDKs, teams lose comparability across models, agents, and workflows. That weakens monitoring, incident review, and policy enforcement because the control plane cannot reason over inconsistent attribute names. Practitioners should treat semantic mapping as part of governance architecture, not as a cosmetic telemetry concern.
A question worth separating out:
Q: How should teams manage AI observability without locking into one SDK?
A: Use framework-agnostic concepts at the platform layer and map each SDK to those concepts at runtime. That lets engineering teams keep their preferred instrumentation while operations teams maintain portability, consistent reporting, and a stable control model across applications.
👉 Read our full editorial: Server-side semantic mapping makes AI observability portable across frameworks
