Notifications
Clear all
Topic starter
21/08/2026 4:51 pm
TL;DR: TrueFoundry’s integration with Pydantic Logfire shows how OpenTelemetry tracing can make AI gateway activity, model routing, and MCP tool calls visible end to end, with spans carrying gen_ai attributes and asynchronous export preserving latency. TruFoundry frames the governance issue as not observability alone but whether identity, tool use, and execution can be audited without changing application code.
NHIMG editorial — based on content published by TruFoundry: Full-Stack LLM Tracing with Pydantic Logfire and TrueFoundry AI Gateway
By the numbers:
- TrueFoundry says its AI Gateway handles 350+ RPS on just 1 vCPU.
- 10ms latency even under load., ay adds about 10ms latency even under load.
Questions worth separating out
Q: How should security teams govern AI gateway traffic that carries prompts and tool calls?
A: Security teams should govern AI gateway traffic as a runtime policy problem, not just a routing problem.
Q: Why do AI gateways create new identity governance concerns?
A: AI gateways sit between users, service accounts, agents, and models, so they become the place where identity, authorisation, and data controls either stay coherent or fragment.
Q: What breaks when organisations rely on observability instead of access control?
A: What breaks is the assumption that telemetry can replace authorization.
Practitioner guidance
- Instrument the gateway as the primary evidence source Use the AI gateway trace stream as the canonical record for model requests, routing decisions, and MCP tool calls.
- Separate telemetry from enforcement Do not assume that seeing a tool call in observability means the call was authorised.
- Review MCP tools as governed access paths Inventory every tool reachable through the gateway, then assign ownership, access scope, and review cadence to each one.
What's in the full article
TruFoundry's full post covers the operational detail this analysis intentionally leaves for the source:
- Step-by-step OTEL Config setup for traces and metrics export to Logfire regional endpoints
- The exact span fields and tfy.span_type values used to separate ChatCompletion, AgentResponse, and MCPGateway traffic
- Logfire Live view and SQL query examples for tracing prompts, tool arguments, and token usage across teams
- The write-token workflow and configuration details needed to stand up the integration securely
👉 Read TruFoundry's integration guide for AI gateway tracing and Logfire →
AI gateway tracing and MCP tool calls: what governance teams miss?
Explore further
21/08/2026 5:09 pm
AI gateway tracing is becoming the evidence layer for non-human identity governance. Once model requests, tool calls, and routing decisions are recorded in standard telemetry, teams can reconstruct what happened without changing every upstream application. That shifts AI governance from code-local logging to infrastructure-level proof, which is more sustainable for large environments. The practitioner conclusion is that observability is now part of identity control design, not just operations.
A few things that frame the scale:
- 92% agree governing AI agents is critical to enterprise security, yet only 44% have implemented any policies to do so, according to AI Agents: The New Attack Surface report.
- Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
A question worth separating out:
Q: What should teams check before using AI gateway traces for investigations?
A: Teams should confirm trace completeness, retention, access restrictions, and correlation across models, providers, and tool calls. If any of those are weak, the record may be too fragmentary to support incident response or compliance review. The goal is a defensible evidence chain, not a dashboard that is easy to read.
👉 Read our full editorial: OpenTelemetry tracing for AI gateways exposes hidden control gaps
