Notifications
Clear all
Topic starter
21/08/2026 4:49 pm
TL;DR: TrueFoundry’s AI Gateway exports OpenTelemetry spans asynchronously to OpenLIT, preserving request-path latency while optionally forwarding prompt, response, and metadata fields into ClickHouse-backed observability pipelines, according to TruFoundry. The governance question is not instrumentation, but where AI request content is allowed to flow, persist, and become queryable.
NHIMG editorial — based on content published by TruFoundry: Exporting TrueFoundry AI Gateway Traces to OpenLIT via OTLP
By the numbers:
- The gateway’s performance profile includes ~10ms latency even under load and 350+ RPS on just 1 vCPU.
- When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes.
Questions worth separating out
Q: How should teams govern AI gateway traces that contain prompt and response data?
A: Treat AI gateway traces as sensitive data flows, not generic logs.
Q: Why do AI observability platforms create new data-governance risk?
A: Because they often centralise information that was previously transient inside the application path.
Q: How do security teams know whether telemetry export is too permissive?
A: Look for content in trace attributes, broad dashboard or SQL access, long retention periods, and uncontrolled downstream copies.
Practitioner guidance
- Restrict exported span attributes Remove prompt, response, and other content-bearing fields from telemetry unless a documented use case requires them.
- Apply trace-store access governance Treat ClickHouse tables containing AI traces as sensitive datasets.
- Define telemetry retention and deletion rules Set retention periods for spans and metrics based on security and compliance needs, not just storage cost.
What's in the full article
TruFoundry's full blog post covers the operational detail this post intentionally leaves for the source:
- Exact OTEL exporter configuration for traces and metrics, including endpoint structure and encoding choices
- ClickHouse table layout and query syntax for trace-level analysis using SpanAttributes
- Helm chart and Kubernetes deployment details for the OpenLIT collector, dashboard, and database
- The internal service DNS names and port mappings used to wire the gateway to the collector
👉 Read TruFoundry's walkthrough of AI Gateway trace export to OpenLIT →
AI gateway trace export to OpenLIT: what changes for governance?
Explore further
21/08/2026 5:08 pm
AI telemetry has become a governance boundary, not a passive by-product. Once prompt and response content are exported into observability pipelines, the security model changes from application protection to data governance. That brings retention, access control, and queryability into scope alongside model operations. Practitioners should treat trace export as a controlled data movement decision, not a logging convenience.
A question worth separating out:
Q: What should teams do before enabling external trace export from an AI gateway?
A: Review which fields are exported, who can read the destination, and whether the destination is isolated from other sensitive workloads. Then test failure behaviour, retention settings, and deletion paths. If those controls are not defined first, external export can create a compliance and exposure problem faster than it improves visibility.
👉 Read our full editorial: TrueFoundry AI Gateway trace export changes observability boundaries
