LangSmith reviews show production AI needs governance, not tracing alone

At a glance

What this is: This is an analysis of why tracing-centric AI observability falls short once LLM applications move into production and governance becomes mandatory.

Why it matters: IAM, NHI, and AI security teams should treat observability as only one layer, because production AI now needs control over outputs, drift, evidence, and policy enforcement.

By the numbers:

• The global observability market reached $5.1 billion in 2021 and projects to $20.5 billion by 2028.
• 90% of IT professionals consider observability important to their business, yet only 26% rate their practice as mature.
• Organizations with advanced observability deployments reduce downtime costs by 90%, dropping from $23.8M to $2.5M annually.

Context

LangSmith is positioned as a developer-focused tracing and evaluation tool for LLM applications, but the governance gap appears once those systems are asked to operate safely in production. Tracing shows what happened; it does not decide what should be allowed, blocked, or audited across multi-step workflows, retrieval pipelines, and external dependencies.

For AI programmes, that gap matters because production risk is no longer limited to model quality. Security teams, compliance leads, and platform owners need runtime controls, monitoring for drift, and evidence that can be mapped to frameworks such as NIST AI RMF and the EU AI Act. In practice, this is where trace-only observability stops being enough.

The article’s starting position is typical of teams that begin with debugging and then discover they also need governance.

Key questions

Q: Why do AI observability tools not provide enough control for production AI systems?

A: AI observability tools tell you what happened, but they usually stop at metrics, alerts, and drift detection. That leaves a gap when a model or agent produces something unsafe, because the system can be warned without being stopped. A control plane closes that gap by enforcing thresholds at runtime and preventing non-compliant outputs from leaving the API boundary.

Q: When should teams move from tracing to runtime governance for AI systems?

A: Teams should make that shift when AI systems can access sensitive data, call external tools, influence customer decisions, or operate across multiple owners and compliance obligations. That is the point where debugging value is no longer enough and enforcement, monitoring, and evidence become operational requirements.

Q: What breaks when AI systems are only protected by pre-release guardrails?

A: Static guardrails fail when the risk appears only after deployment, during live inference, tool use, and action execution. The model can pass staging checks yet still drift, misuse privileges, or act on manipulated inputs in production. Practitioners need controls that enforce policy at the moment of action, not just documentation that the system was tested before release.

Q: How should security teams govern agentic AI as it moves into production?

A: Security teams should govern agentic AI as a class of non-human identity, not as a generic application feature. That means assigning ownership, scoping permissions tightly, logging every tool action, and revoking access on a defined lifecycle. Production rollout should require clear approval points for high-risk actions and continuous monitoring for drift.

Technical breakdown

Why trace-only observability stops at the wrong layer

Tracing captures the sequence of model calls, tool invocations, retrieval steps, and latency, which makes it valuable for debugging prompt chains and isolating regressions. The limitation is that traces are retrospective records, not policy enforcement points. Once an LLM workflow includes external APIs, agent steps, or retrieval-augmented generation, the control problem shifts from visibility to runtime decisions about allow, deny, or escalate. That is the point where observability becomes necessary but insufficient.

Practical implication: treat tracing as investigation support, not as the control that prevents unsafe behavior.

Production AI adds governance obligations that developers do not own alone

Production AI creates shared responsibility across engineering, security, compliance, and risk functions. A developer tool may help test prompts, but enterprise deployment also needs drift monitoring, safety checks, evidence capture, and framework mapping. These requirements are especially hard when a system spans LangChain apps, third-party services, and multi-step agent workflows. Governance is therefore less about dashboards and more about proving that the system remains within approved risk boundaries as it changes.

Practical implication: align AI observability with governance ownership before production handoff, not after incidents begin.

Why runtime guardrails matter more than offline evaluation alone

Offline evaluation is useful for comparing versions and catching regressions, but it does not stop prompt injection, data exfiltration, or harmful outputs in the moment of execution. Runtime guardrails sit between model behavior and downstream impact, which is why they matter when AI systems can take sequential actions or invoke tools. For regulated deployments, this is also where auditability and policy enforcement intersect. A system that cannot block harmful output at runtime relies too heavily on post hoc review.

Practical implication: require live enforcement controls for any AI workflow that can trigger real-world actions or expose sensitive data.

Threat narrative

Attacker objective: The objective is to make AI behavior produce unsafe, unauditable, or policy-breaking outcomes inside production workflows.

1. Entry occurs through normal AI application use, where prompt chains, retrieval steps, or agent workflows are executed in production.
2. Credential or data exposure follows when the workflow can surface sensitive context, misuse external APIs, or propagate unsafe outputs into downstream systems.
3. Impact is governance failure rather than a classic breach: harmful responses, unbounded drift, and weak auditability undermine trust and regulatory readiness.

Breaches seen in the wild

• McKinsey AI platform breach: McKinsey AI platform hack exposed 46M chats and sensitive data.
• Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Trace visibility is not governance: the market keeps treating observability as the answer when the real question is whether AI systems can be constrained at runtime. Traces help teams explain failures after they occur, but production AI now needs policy enforcement, audit evidence, and drift controls to stay inside approved boundaries. For practitioners, the control question is no longer whether a system can be inspected, but whether it can be safely governed while it runs.

Production AI creates an identity and authority problem, not just a model-quality problem: once agents, tools, and third-party APIs enter the workflow, the system is making decisions through delegated authority. That means security teams have to think about who or what is authorised to act, what data it can reach, and how its behavior is constrained over time. For practitioners, AI observability must connect to identity, privilege, and runtime policy, not sit beside them as a separate dashboard.

Governance debt is the better named concept here: teams can accumulate traces faster than they can build the controls needed to prove safety, compliance, and accountability. That debt becomes visible when regulated use cases, distributed ownership, and multi-step workflows arrive at the same time. For practitioners, the issue is not adding more logs, but reducing the gap between inspection and enforcement before production scale hardens the wrong operating model.

Compliance mapping is becoming a design requirement, not a documentation exercise: frameworks such as NIST AI RMF and the EU AI Act increasingly shape how enterprises approve and monitor AI systems. If evidence generation is manual, governance becomes fragile as scope expands across teams and vendors. For practitioners, the priority is to make compliance evidence part of the control plane, not a post-deployment reporting task.

Agentic workflows widen the control surface faster than traditional monitoring can absorb: multi-step reasoning, retrieval dependencies, and external services create failure paths that single-call monitoring misses. This is where agent identity, authorization, and output control converge. For practitioners, the only sustainable model is one that treats agent behavior as governed execution, not just observable execution.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Authorisation Guide

What this signals

Production AI programmes will increasingly be judged on enforcement, not instrumentation. Teams that stop at tracing will still know what happened, but they will not be able to prove that harmful behavior was prevented before it reached a downstream system.

Governance debt: the real risk is the accumulation of observability without control. As workflows spread across teams, the cost of retrofitting guardrails, audit evidence, and compliance mapping rises faster than the cost of designing them in from the start.

AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026. That makes runtime governance a practical control problem, not a theoretical maturity issue.

For practitioners

• Define the production control boundary Separate debugging telemetry from enforcement points so teams know which controls can inspect behavior and which can stop it before downstream impact.
• Map AI systems to governance obligations Inventory which workflows need drift monitoring, audit evidence, compliance mapping, and approval workflows before they move beyond experimentation.
• Add runtime guardrails to high-risk workflows Block unsafe outputs, prompt injection, and data exfiltration at execution time for AI systems that can access sensitive data or trigger actions.
• Tie observability to identity and privilege Track which agent, application, or delegated workflow can act, what it can reach, and how that authority is constrained across tool calls.

Key takeaways

• Tracing is valuable for AI debugging, but production governance requires runtime controls that can block unsafe behavior, not just explain it after the fact.
• The article shows a common maturity gap: teams can observe more than they can enforce, especially once workflows include tools, retrieval, and delegated actions.
• The most effective response is to pair observability with guardrails, evidence capture, and identity-aware authority controls before AI systems scale.

Key terms

• AI observability: AI observability is the ability to see how AI systems are being used, what information they process, and what actions they trigger. In security programmes, it extends beyond uptime or model quality to runtime visibility, policy enforcement, and audit evidence across human and agent-driven use cases.
• Runtime Guardrail: A control applied while an AI agent is operating, not just during configuration or review. Guardrails can block dangerous tool calls, require approval for sensitive actions, or stop data leakage before it reaches systems or users.
• Drift Monitoring: Drift monitoring tracks whether inputs, embeddings, or outputs are changing over time in ways that can degrade model performance. It is an early-warning control that helps teams spot behaviour shifts before they become visible business or security failures.
• AI Governance: AI governance is the set of controls used to discover, classify, approve, restrict, monitor, and revoke AI-enabled access. It connects identity, data, and policy so organisations can manage what AI can reach, what it can share, and when it should be stopped.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and machine identity security. It helps practitioners connect identity controls to broader security and governance programmes.