Topic starter
21/08/2026 4:51 pm
TL;DR: TrueFoundry’s integration with Pydantic Logfire shows how OpenTelemetry tracing can make AI gateway activity, model routing, and MCP tool calls visible end to end, with spans carrying gen_ai attributes and asynchronous export preserving latency. TruFoundry frames the governance issue as not observability alone but whether identity, tool use, and execution can be audited without changing application code.
Editorial analysis by NHI Mgmt Group, based on content published by TruFoundry: “Full-Stack LLM Tracing: Pydantic Logfire and TrueFoundry AI Gateway”.
By the numbers:
- TrueFoundry says its AI gateway handles 350+ RPS on just 1 vCPU.
Key questions
Q: How should teams govern AI gateways that route model and tool traffic?
A: Teams should treat the gateway as the control boundary for identity, spend, logging, and policy enforcement.
Q: Why do AI gateways need traceable tool-call evidence?
A: Because model output alone rarely explains how a task was actually executed.
Q: What breaks when AI gateway spans do not preserve MCP context?
A: The audit trail becomes fragmented.
Practitioner guidance
- Define the gateway trace schema Map the span fields you need for routing, model selection, token usage, and MCP tool calls before rollout so the trace can support later review.
- Validate asynchronous export reliability Test the queue, exporter service, and OTLP endpoint under load so trace delivery failure is visible before you depend on it for audit evidence.
- Correlate tool calls with parent requests Require parent-child span linkage for every MCP interaction so investigators can reconstruct which request led to which tool execution.
Bottom line: AI gateway tracing can expose model routing and MCP tool use, but only if the span model preserves enough context to reconstruct execution.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
21/08/2026 5:09 pm
AI gateway tracing is becoming a governance control, not just an operations feature: once the gateway brokers model access and MCP tool calls, the trace is the evidence trail for execution decisions. That shifts observability into the identity-security domain because the organisation needs to prove which request led to which tool action and under what context. Practitioners should treat the trace schema as part of the control design, not as an afterthought.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: How do AI gateway traces differ from ordinary application logs?
A: Traces preserve causal sequence across the request path, while logs usually record isolated events. For AI gateways, that distinction matters because routing decisions, model responses, and tool use need to be correlated in time and context. A trace can show the path; disconnected logs usually cannot.
👉 Read our full editorial: OpenTelemetry tracing for AI gateways exposes hidden control gaps