Notifications
Clear all
Topic starter
21/08/2026 4:48 pm
TL;DR: TrueFoundry’s walkthrough shows how its AI Gateway emits OpenTelemetry spans for every request, including MCP tool calls, and TruFoundry exports them asynchronously to Honeycomb so teams can trace model, tool, and response activity without adding request-path latency. The governance issue is not observability alone but what trace content exposes about prompt data, access headers, and agentic workflows.
NHIMG editorial — based on content published by TruFoundry: Exporting TrueFoundry AI Gateway Traces to Honeycomb with OpenTelemetry
By the numbers:
- TrueFoundry says its gateway can handle 350+ RPS on just 1 vCPU.
Questions worth separating out
Q: How should security teams govern AI gateway traffic that carries prompts and tool calls?
A: Security teams should govern AI gateway traffic as a runtime policy problem, not just a routing problem.
Q: Why do AI gateways create new authorization risks for NHI governance?
A: Because they concentrate many decisions into one control point while still relying on coarse scopes in many deployments.
Q: What breaks when trace export is left outside security review?
A: Teams can end up with a second data path that bypasses the controls applied to the application runtime.
Practitioner guidance
- Define a trace data minimization policy Classify which span attributes may leave the AI gateway, then strip prompts, responses, and any secret-bearing fields before export.
- Separate production and staging datasets Use distinct dataset routing for production, staging, and MCP traffic so analysts do not inherit a mixed trace corpus.
- Protect export credentials as machine identities Treat the Honeycomb ingest key and any similar headers as NHI secrets with rotation, scope restriction, and monitoring.
What's in the full article
TruFoundry's full article covers the implementation details this post intentionally leaves at the architecture level:
- The exact OpenTelemetry span fields emitted by the gateway, including the gen_ai.* and tfy.* attributes
- The full Honeycomb configuration surface, including endpoint, dataset routing, and authentication header values
- The request-to-trace lifecycle showing how NATS decouples export from inference availability
- The differences between TrueFoundry export behavior and the OpenTelemetry Collector's OTLP HTTP path handling
👉 Read TruFoundry's walkthrough of AI Gateway trace export to Honeycomb →
AI Gateway traces and MCP payloads: what governance teams should watch?
Explore further
21/08/2026 5:07 pm
Observability has become an identity control surface in AI gateways. Once traces include prompts, headers, tool calls, and model routing, telemetry is no longer just an operations artifact. It becomes evidence of who or what acted, on which resource, and through which delegated path. That makes OpenTelemetry design relevant to IAM, PAM, and NHI governance, especially where MCP servers and AI agents are involved. Practitioners should treat exported trace fields as governed identity data, not generic logs.
A question worth separating out:
Q: Should organisations separate observability access from platform access?
A: Yes. The people who need to operate an AI gateway do not always need access to raw prompts, full tool payloads, or every production trace. Separate those permissions so that troubleshooting, security review, and data inspection remain distinct roles. That reduces unnecessary exposure while still preserving investigatory value.
👉 Read our full editorial: TrueFoundry AI Gateway traces show where AI observability meets governance
