Should organisations use OpenTelemetry GenAI conventions, OpenInference, or both in mixed AI environments?

Use OpenInference when you control application instrumentation and want an established AI-specific schema. Use OpenTelemetry GenAI conventions when frameworks or managed platforms already emit gen_ai spans or when OTLP export is your only control point. In mixed environments, using both can preserve existing traces while normalizing compatible telemetry into a common operational view.

Why This Matters for Security Teams

For mixed AI environments, the real issue is not which telemetry label is fashionable, but whether security teams can reliably reconstruct what an AI system did, what data it touched, and which control plane produced the record. That matters for incident response, model risk review, and auditability. OpenTelemetry GenAI conventions and OpenInference both aim to make AI activity observable, but they are not always interchangeable. The risk is fragmenting traces across tools, runtimes, and managed services so that detections, investigations, and governance checks lose continuity. Current guidance on AI observability and risk management, including the NIST AI 600-1 GenAI Profile, points toward consistent measurement and traceability rather than a single mandated schema.

Security teams also need to distinguish operational telemetry from assurance evidence. A trace that captures prompts, tool calls, retrieval steps, and responses can support investigations, but only if identifiers are stable and retention is governed. In practice, telemetry choices affect whether alerts can be correlated with identity, workload, or API-layer events. In practice, many security teams encounter missing AI telemetry only after an adverse prompt injection, data exposure, or agent misuse has already occurred, rather than through intentional control design.

How It Works in Practice

OpenTelemetry GenAI conventions are most useful when the environment already speaks OTLP or when a platform, framework, or service mesh is emitting standardized span attributes for model requests, prompts, embeddings, and tool activity. OpenInference is stronger when the application team can instrument the AI workflow directly and wants a schema purpose-built for AI application traces, including retrieval steps and model interactions. In mixed environments, the practical pattern is usually to preserve the richest source schema at the edge and translate only where necessary into a shared backend representation.

That means the integration design should answer three questions first: where telemetry is created, which attributes are required for detection or governance, and where normalization should happen. For example, a managed LLM service may emit OpenTelemetry-native traces at the platform layer, while a custom RAG application may emit OpenInference spans inside the app. Security teams often map both into a common observability backend so analysts can pivot from request to model to tool call without losing context.

• Preserve source fidelity first, then normalize for search, correlation, and alerting.
• Keep prompt, retrieval, tool, and response metadata consistent where policy permits.
• Define retention and redaction rules before telemetry reaches the shared pipeline.
• Validate that trace IDs and service names remain stable across orchestration layers.

When AI systems touch secrets, customer data, or privileged tools, telemetry should also be aligned with identity and access logging so investigators can tell which human, service, or agent initiated the action. Best practice is evolving here, but the operational goal is clear: create a chain of custody from request to model output to side effect. These controls tend to break down when managed platforms, custom code, and serverless agent runners all emit different span taxonomies because correlation logic becomes brittle and analysts lose end-to-end context.

Common Variations and Edge Cases

Tighter telemetry standardisation often increases engineering overhead, requiring organisations to balance observability depth against instrumentation effort and privacy constraints. That tradeoff becomes sharper in regulated or multi-tenant environments where prompts and retrieval content may contain personal data, secrets, or proprietary context.

There is no universal standard for this yet, so organisations should avoid treating dual support as automatic best practice. Use both approaches when the environment is genuinely mixed, but document which system is authoritative for each span type. A platform team may prefer OpenTelemetry conventions for infrastructure-consistent export, while application teams may prefer OpenInference for richer AI semantics. The important point is to prevent duplicate telemetry from creating false positives or hiding actual abuse behind repeated spans.

Edge cases also appear when agents act autonomously across tools. In those scenarios, telemetry should capture the initiating identity, the decision path, and any external callouts in a way that supports OWASP guidance for LLM risk patterns and internal governance reviews. If the environment includes vendor-hosted models or opaque orchestration layers, teams may only see partial span data, so the goal shifts from perfect normalization to defensible coverage. For broader model governance, MITRE and telemetry guidance from OpenTelemetry are useful reference points, but local policy should decide what is required for detection, audit, and privacy control.

Related resources from NHI Mgmt Group

• Should organisations use just-in-time access for AI development environments?
• When should organisations use AI to help manage NHIs?
• How can organisations govern AI agents that use service accounts and tokens?
• Should organisations allow AI agents to use hardcoded secrets?