Notifications
Clear all
Topic starter
21/08/2026 4:50 pm
TL;DR: AI Gateway trace export shows that OpenTelemetry spans can carry full prompt and response bodies, and TruFoundry’s explanation says asynchronous export keeps telemetry off the request path and separates internal storage from SigNoz Cloud ingestion. That design makes observability possible without latency penalties, but it also turns telemetry export settings into an access-control decision, not just an operations choice.
NHIMG editorial — based on content published by TruFoundry: Exporting TrueFoundry AI Gateway traces to SigNoz via OTLP
By the numbers:
- The gateway handles 350+ RPS on just 1 vCPU and advertises ~10 ms latency even under load.
Questions worth separating out
Q: How should security teams govern AI gateway traffic that carries prompts and tool calls?
A: Security teams should govern AI gateway traffic as a runtime policy problem, not just a routing problem.
Q: Why do AI gateways create new identity governance concerns?
A: AI gateways sit between users, service accounts, agents, and models, so they become the place where identity, authorisation, and data controls either stay coherent or fragment.
Q: What breaks when trace export is not governed as part of the application design?
A: Teams lose control over where prompts, outputs, and routing details are stored, who can query them, and whether they leave the intended trust boundary.
Practitioner guidance
- Inventory span fields before enabling external export Review every attribute the gateway can emit, including tfy.input, tfy.output, and routing metadata, then classify which fields are allowed to leave the runtime and which must be removed or masked.
- Use trace exclusion for sensitive workloads Enable the Exclude Request Data toggle for workloads that need latency visibility but should not transmit prompt or response content to SigNoz or any other external backend.
- Monitor exporter loss as a control failure Track dropped spans, queue backlog, and ingestion failures so the team can tell when audit visibility has degraded even though request latency remains normal.
What's in the full article
TruFoundry's full article covers the operational detail this post intentionally leaves for the source:
- Exact OTLP exporter settings for traces and metrics, including endpoint format, protocol, and headers.
- SigNoz ingestion and storage pipeline details, including how spans flow through the shared gateway into per-tenant ClickHouse.
- Field-by-field explanation of the span attributes such as tfy.span_type, tfy.input_short_hand, and gen_ai semantic conventions.
- Practical configuration examples for filtering by service.name and destination in SigNoz explorers.
👉 Read TruFoundry's explanation of AI Gateway trace export to SigNoz →
AI gateway traces and request content: are your export controls enough?
Explore further
21/08/2026 5:08 pm
Telemetry export is now a governance surface, not an operations detail. AI gateways increasingly emit spans that can include full prompts and completions, which means observability platforms can become accidental data processors. That changes the control model from simple monitoring to content-aware governance, especially where secrets, customer data, or regulated information may appear in traces. Practitioners should treat trace export like any other sensitive data flow, with policy, minimisation, and auditability built in.
A question worth separating out:
Q: Should organisations use external observability platforms for AI workloads by default?
A: Only if they can enforce field-level minimisation, tenant separation, and credential governance for the export path. External observability is workable, but it should be a deliberate choice with explicit data handling rules, not a default assumption that all spans are safe to ship.
👉 Read our full editorial: AI gateway telemetry exposes request content unless export is governed
