Why does OpenTelemetry not fully solve AI observability governance?

Why This Matters for Security Teams

OpenTelemetry is useful for collecting traces, metrics, and logs, but governance depends on whether those signals can be interpreted consistently across models, tools, and deployment patterns. For AI systems, the challenge is not just transport fidelity. It is whether the telemetry can answer questions about prompt sources, model versioning, tool use, data lineage, and policy exceptions in a way that supports audit and risk decisions. The NIST Cybersecurity Framework 2.0 is helpful here because it treats observability as part of broader governance, not a standalone logging exercise.

Security teams often assume that if an AI workflow emits telemetry, governance is solved. That breaks down quickly when one framework labels a field one way, another uses a different schema, and a third omits the context needed to explain why a tool call happened. The result is fragmented evidence, inconsistent dashboards, and policy enforcement that depends on manual interpretation rather than control design. In practice, many security teams encounter governance gaps only after an incident review exposes missing lineage, rather than through intentional observability planning.

How It Works in Practice

OpenTelemetry can be an effective collection layer for AI applications, but it does not define the governance model around the data it carries. Practitioners still need a control plane that standardises semantic meaning, retention rules, access boundaries, and decision thresholds. Without that layer, the same trace may show a model invocation, a retrieval step, and a tool action, yet remain unusable for comparing risk across environments or proving that an AI workflow respected policy.

Operationally, teams usually need to build a translation layer between framework-specific attributes and an internal governance schema. That schema should cover model identity, prompt origin, RAG source references, tool invocation context, human approval status, and escalation outcomes. For AI security teams, this is also where OWASP guidance for LLM applications becomes useful, because prompt injection, insecure tool use, and excessive agency often show up first in telemetry patterns before they are visible in business logic.

• Normalise event names so the same control is measured the same way across frameworks.
• Preserve model and prompt lineage so investigators can reconstruct what the system saw and produced.
• Tag tool calls and external data access separately from ordinary application spans.
• Map telemetry fields to policy decisions, not just to dashboards.
• Restrict access to sensitive traces, since observability data can expose prompts, secrets, and user content.

For agentic workflows, this governance layer becomes even more important because execution authority can shift dynamically. The MITRE ATLAS adversarial AI framework is useful for thinking about how an attacker might manipulate model outputs, context, or tool interactions, while the OWASP Agentic AI Top 10 helps teams identify where autonomy increases exposure. These controls tend to break down when telemetry is collected from mixed model stacks with no shared schema, because the governance layer cannot reliably correlate behaviour across inference services, agents, and downstream tools.

Common Variations and Edge Cases

Tighter observability usually increases operational overhead, requiring organisations to balance richer AI oversight against latency, storage, and privacy constraints. That tradeoff becomes more pronounced when telemetry includes user prompts, retrieval content, or tool outputs that may contain regulated or sensitive data. Best practice is evolving here, and there is no universal standard for how much AI context should be logged by default.

One common edge case is hybrid environments where one team uses a mature tracing library and another uses a custom agent framework that emits only partial metadata. Another is when governance teams want model-level comparisons, but the telemetry is optimised for application debugging rather than policy evidence. In those cases, OpenTelemetry can still be part of the solution, but it should be paired with explicit control definitions, schema governance, and review workflows that align with enterprise risk management. That is also where a broader ai governance lens, such as the NIST AI Risk Management Framework, helps connect engineering telemetry to accountability and oversight.

For organisations handling cross-border or regulated AI use cases, current guidance suggests treating observability data as governance evidence, not just operational noise. That means defining who can see traces, how long they are retained, and how exceptions are escalated when telemetry conflicts with declared policy. OpenTelemetry is valuable, but it does not resolve the harder question of what the data means or who is responsible for acting on it.

Related resources from NHI Mgmt Group

• Why do observability platforms fail to solve AI governance on their own?
• Why does shift-left security not fully solve AI agent risk?
• What is the difference between AI observability and AI governance?
• What breaks when organisations rely only on observability for AI governance?