TL;DR: TrueFoundry’s walkthrough shows how its AI Gateway emits OpenTelemetry spans for every request, including MCP tool calls, and TruFoundry exports them asynchronously to Honeycomb so teams can trace model, tool, and response activity without adding request-path latency. The governance issue is not observability alone but what trace content exposes about prompt data, access headers, and agentic workflows.
At a glance
What this is: TruFoundry describes an AI Gateway tracing design that captures request, model, and MCP tool activity end to end, while also exposing governance questions around prompt data and header leakage in exported spans.
Why it matters: IAM, NHI, and AI governance teams need to understand that observability pipelines can become data-exposure paths if trace payloads include secrets, prompts, or tool-call context.
By the numbers:
- The gateway handles 250+ requests per second with approximately 3 ms of added latency.
- The product page says the gateway can handle 350+ RPS on just 1 vCPU.
Context
AI gateway observability is useful only when teams can see request flow, model selection, and tool use without breaking production performance. The governance gap appears when trace data carries prompt content, headers, and agentic tool payloads into downstream systems that were not designed as sensitive-data repositories.
This article sits at the intersection of AI infrastructure and identity governance because the gateway authenticates requests, resolves model access, and traces MCP tool calls in the same execution path. That makes trace export a security boundary, not just a telemetry choice.
Key questions
Q: How should security teams govern AI gateway traffic that carries prompts and tool calls?
A: Security teams should govern AI gateway traffic as a runtime policy problem, not just a routing problem. Inspect prompts, responses, and tool calls before they reach downstream systems, and make sure the gateway logs enough context to show what was sent, what was returned, which policy applied, and what action followed.
Q: Why does asynchronous trace export change the risk profile of an AI gateway?
A: Because it moves telemetry handling out of the request path and into a separate data store. That protects latency, but it also creates another place where sensitive AI content can persist, be indexed, and be reused for reporting or alerting. The risk increases whenever observability data contains business or personal information.
Q: What are the signs that AI observability is not giving teams enough operational insight?
A: Warning signs include blind spots in LLM calls, retrieval pipelines, multistep agents, and downstream application logic. If teams cannot trace agent to agent interactions, tool calls, or message passing sequences, they will miss coordination failures and protocol deviations. Weak observability usually shows up as incidents that are discovered too late to explain or contain confidently.
Q: How should teams govern MCP tracing in agentic workflows?
A: Apply the same controls you would use for delegated machine access. Limit which tool-call attributes are exported, separate production and staging datasets, and verify that the trace does not expose enough detail to turn observability into an attack aid.
Technical breakdown
OpenTelemetry spans in the AI gateway request path
The gateway creates a span tree for each request, covering inbound handling, JWT-based authentication, model resolution, provider invocation, and streaming response handling. That matters because the trace is not just a log line after the fact. It is a structured record of control flow, and in an AI gateway that record can include user metadata, model identifiers, latency, token counts, and tool-call context. If trace fields are rich enough to reconstruct work patterns, they become governance objects in their own right, not mere observability data.
Practical implication: Treat trace attribute design as part of data governance, not only telemetry engineering.
Asynchronous export changes the security boundary for traces
TruFoundry publishes spans to NATS after the request completes, then a background exporter forwards them to an OTLP backend. This decouples telemetry delivery from request serving, which protects availability, but it also means trace data is copied into another system outside the serving path. That second hop changes who can access the data, how long it persists, and what controls apply. When prompt text or tool payloads are exported, the observability backend effectively becomes part of the sensitive data lifecycle.
Practical implication: Classify the telemetry backend as a governed destination for sensitive AI interaction data.
MCP tool-call tracing expands identity and payload exposure
The same tracing model applies to MCP gateway activity, where each tool invocation records the calling user, the MCP server, the tool name, the request, the response, and latency. That gives strong diagnostic value, but it also makes tool traffic visible at a granularity that can reveal access paths, delegated actions, and embedded secrets if payloads are not controlled. In identity terms, the trace becomes a record of who or what exercised tool authority, which is why NHI and agentic AI governance have to reach into the tracing layer.
Practical implication: Apply least-data principles to MCP traces and decide which fields are safe to export by default.
Threat narrative
Attacker objective: The attacker aims to mine exported traces for prompts, headers, and tool-call context that can be reused for access abuse or workflow reconstruction.
- Entry occurs when an AI gateway or MCP flow records sensitive request context, including prompts, headers, or tool payloads, into traces that are exported outside the serving path.
- Escalation follows when the observability backend indexes those fields and makes them broadly queryable, turning diagnostic data into a high-value exposure surface.
- Impact is the disclosure of prompt content, access headers, or tool-call details that can help an attacker reconstruct AI workflows or reuse credentials.
Breaches seen in the wild
- Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.
- McKinsey AI platform breach: McKinsey AI platform hack exposed 46M chats and sensitive data.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
Trace data has become a governance surface, not just an observability artifact. In AI gateways, the same telemetry that helps operators debug latency can also capture prompts, headers, model identifiers, and tool payloads. That makes the export destination part of the control environment, not a neutral sink. Practitioners should treat every exported span attribute as a potential policy decision about disclosure, retention, and access.
AI gateway tracing creates a new class of identity-adjacent risk for NHIs and agents. When an MCP tool call records the calling user, the server, and the tool name in the same trace tree, the organisation gains useful lineage but also exposes delegated authority paths. This is where identity governance meets telemetry: the trace can reveal who exercised machine authority, but it can also leak the evidence needed to misuse it. Governance teams should scope telemetry with the same discipline they apply to credentials.
Least-data for traces is now a control pattern, not a nice-to-have. The article’s optional request-data exclusion is the right design signal, because it acknowledges that prompt text and response text are often more sensitive than the infrastructure metadata around them. The named concept here is observability leakage boundary: the point at which useful trace content becomes unbounded data exposure. Practitioners should define that boundary before exporting traces beyond the gateway.
Asynchronous export improves resilience, but it does not reduce confidentiality obligations. Decoupling export from the request path protects availability, yet it also makes trace replication easier to forget. The more transparent the AI control plane becomes, the more important it is to decide which fields are allowed to leave the environment at all. For enterprise AI, telemetry governance must sit alongside access governance.
From our research library:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
- Read next: AI Agent Observability, Audit and Incident Response Guide
What this signals
Observability leakage boundary: AI gateways are now creating traces that can carry prompt content, access headers, and tool payloads into downstream systems. That means security teams need a clear line between operational telemetry and data that must remain inside the serving environment.
Trace export is becoming a governance decision. As asynchronous pipelines spread, the question is no longer whether tracing is available, but which fields are allowed to leave the gateway at all. That policy belongs with identity and AI governance, not only with platform engineering.
The broader signal is that AI infrastructure is leaking faster than core model providers: AI-related credential leaks surged 81.5% year-over-year in 2025, according to the State of Secrets Sprawl 2026. Teams should assume the observability layer can become part of the secrets exposure surface unless export boundaries are explicit.
For practitioners
- Define a trace data classification policy Classify prompt text, response text, headers, tool payloads, and model identifiers separately so export decisions are explicit rather than incidental.
- Use field-level redaction for AI gateway spans Strip tfy.input, tfy.output, and any sensitive header fields before export when those values are not required for operational debugging.
- Separate observability backends from sensitive serving data Treat the OTLP destination as a governed system of record with its own access controls, retention rules, and audit logging.
- Review MCP trace content for delegated authority exposure Check whether tool-call spans reveal enough context to reconstruct who invoked which tool, against which server, and with what data.
Key takeaways
- AI gateway tracing can improve operational visibility while simultaneously expanding the surface where prompt content, headers, and tool payloads may be exposed.
- The key governance issue is not whether traces exist, but which fields are permitted to leave the serving boundary and be indexed elsewhere.
- Teams should treat observability backends as governed data destinations and apply redaction, access control, and retention rules accordingly.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10 and OWASP Agentic AI Top 10 address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Non-Human Identity Top 10 | NHI-02 — Secret Leakage | Traces can export prompts, headers, and payloads that function like sensitive secrets. |
| NHI-10 — Human Use of NHI | MCP traces reveal how people invoke machine authority through AI workflows. | |
| Recommendation — Redact sensitive trace fields before export and treat exported spans as governed data. Review traces for human-mediated machine access and limit what delegation evidence leaves the gateway. | ||
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Agent and tool traces can expose delegated privilege paths in agentic workflows. |
| Recommendation — Constrain exported agent traces to the minimum data needed to investigate privilege use. | ||
| NIST AI RMF | GOVERN — AI Governance and Accountability | Export policy and trace handling are governance decisions for AI systems. |
| Recommendation — Define governance rules for AI telemetry, retention, and data-sharing boundaries. | ||
| NIST CSF 2.0 | PR.DS-01 — Data-at-Rest is Protected | Exported traces become stored data that must be protected at rest and in transit. |
| Recommendation — Apply data protection controls to trace stores and backend query access. | ||
Key terms
- Observability Leakage Boundary: The observability leakage boundary is the point at which telemetry stops being harmless diagnostics and starts becoming sensitive data exposure. In AI gateways, this boundary is crossed when traces include prompts, headers, responses, or tool payloads that can be queried outside the serving environment.
- Trace Attribute: A key-value field attached to a trace or span. In AI systems, attributes can expose request bodies, outputs, model identifiers, or routing decisions, making attribute design a direct input into privacy, access control, and retention policy.
- Delegated Machine Access: Access exercised by a non-human actor on behalf of a human or another system. The important issue is not only who requested the access, but how far the delegated actor can chain actions once runtime execution begins.
- MCP Trace Exposure: MCP trace exposure is the disclosure risk created when tool-call telemetry includes server names, tool names, request bodies, or response bodies. It matters because the trace can reveal how an AI system interacts with tools, what data it touched, and where control boundaries sit.
Deepen your knowledge
NHI Mgmt Group covers identity security, NHI governance, and agentic AI through the NHI Foundation Level course, the industry's only accredited NHI security programme. It helps practitioners build governance habits that scale across access, telemetry, and delegated machine authority.
Published by the NHIMG editorial team on August 21, 2026.
Updated on October 12, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org