Topic starter
21/08/2026 4:43 pm
TL;DR: AI observability and governance break when teams cannot reconcile tokens, model names, inputs, outputs, and span types across conventions, according to Fiddler. The practical issue is not collection but semantic consistency: without it, platform teams cannot compare, alert, or enforce reliably across AI stacks.
Editorial analysis by NHI Mgmt Group, based on content published by Fiddler: “Server-Side Semantic Mapping for Consistent AI Observability Across Frameworks”.
Key questions
Q: How should teams standardise AI observability across multiple frameworks?
A: Create a shared semantic mapping layer that normalises framework-specific attributes into common concepts such as tokens, model identity, inputs, outputs, and span types.
Q: Why does OpenTelemetry not fully solve AI observability governance?
A: OpenTelemetry standardises transport, not meaning.
Q: What are the signs that AI telemetry semantics are failing?
A: Look for dashboards that disagree across teams, metrics that cannot be compared between frameworks, and alerts that fire on one convention but not another.
Practitioner guidance
- Define a shared AI telemetry semantic layer Map tokens, model identity, inputs, outputs, span types, and evaluator signals to platform-wide concepts before they feed dashboards or policy logic.
- Preserve raw telemetry attributes for traceability Store the original attribute key alongside the resolved concept so engineers can debug framework-specific behaviour without losing the standardised view.
- Standardise span typing across agent workflows Classify model calls, tool actions, retrieval steps, guardrails, and evaluators with the same span taxonomy across frameworks and SDKs.
Bottom line: AI observability can fail even when telemetry is collected correctly if frameworks do not agree on what the data means.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
21/08/2026 5:03 pm
Semantic observability is becoming a governance control, not just a telemetry convenience. Once AI systems span multiple SDKs and frameworks, the real risk is not absence of logs but inconsistent meaning across logs, traces, and metrics. A platform can collect everything and still fail to support reliable comparison, alerting, or enforcement if the same concept is named differently in each convention. The practitioner conclusion is straightforward: governance depends on shared semantics before it depends on analytics.
A few things that frame the scale:
- 7% of security leaders admit they do not know how often their AI systems are making autonomous changes to infrastructure, according to the 2026 Infrastructure Identity Survey.
A question worth separating out:
A: Centralise it when teams use more than one framework or expect frameworks to change over time. A platform-level mapping layer reduces code churn, keeps observability portable, and avoids forcing every application owner to maintain its own interpretation rules for the same AI behaviour.
👉 Read our full editorial: Server-side semantic mapping makes AI observability portable across frameworks