LangChain, LangGraph and LangSmith: the governance gap in agent stacks

At a glance

What this is: TruFoundry maps LangChain, LangGraph and LangSmith to different layers of an agent stack, showing that build, orchestration and observability are distinct governance problems.

Why it matters: IAM and AI security teams need to treat agent frameworks as a control boundary issue, because tool access, stateful execution and tracing each create different governance obligations.

Context

LangChain, LangGraph and LangSmith sit in different layers of the modern agent stack, so comparing them only as development tools misses the real governance problem. Once agent workflows branch, loop and call external tools, control moves from code composition into access, state, observability and lifecycle management.

For identity and AI governance teams, the important question is not which framework is preferred but where policy enforcement belongs. The article’s subject is typical of a broader pattern in production AI: capability is moving faster than controls, and teams are discovering that orchestration and traceability do not automatically provide authorisation or accountability.

Key questions

Q: How should teams govern AI agents that span LangChain, LangGraph and LangSmith?

A: Treat the stack as three different control layers. Use the framework layer for composition, the graph layer for stateful execution, and the observability layer for investigation, but enforce authorisation separately from tracing. If tool access is not scoped before execution, better orchestration only makes misuse easier to perform and harder to contain.

Q: Why do stateful agent frameworks create more operational risk than simpler LLM workflows?

A: Stateful agent frameworks increase risk because each step can influence later decisions, so errors, bad inputs, or poor tool choices can compound across the run. Persistence and memory improve capability, but they also make behavior harder to predict and audit. That means teams need tighter traceability, stronger controls around tools, and explicit checks on intermediate state.

Q: What are the signs that AI agent lifecycle controls are failing?

A: Common signals include undocumented agents, unclear ownership, persistent tokens, broad permissions that exceed task scope, and retired agents that still have live integrations. These indicators show that identity state is drifting away from governance and that offboarding is not closing access cleanly.

Q: What is the difference between agent observability and agent authorisation?

A: Agent observability shows what an identity did after access began, while authorisation decides whether that identity should have had access and how much. In production, authorisation is the preventive control and observability is the detective one. They solve related problems, but they are not interchangeable.

Technical breakdown

LangChain as the application composition layer

LangChain is best understood as the composition layer for LLM applications. Its Runnable model, prompt templates, retrievers, tools and parsers let developers wire together linear pipelines for chat, RAG and simple tool calls. That design works when the system follows a predictable path, but it does not persist state across a complex decision loop. From a governance perspective, that means LangChain helps define the app structure, not the control envelope around what the app is allowed to do.

Practical implication: teams should not confuse a clean LLM pipeline with a governed agent, because composition alone does not enforce tool or data access policy.

LangGraph as stateful orchestration for agent workflows

LangGraph adds explicit state, conditional routing, checkpointing and loops, which makes it suitable for agents that must decide what happens next based on prior actions. In practice, it turns the agent into a graph of nodes and edges rather than a fixed sequence of calls. That is powerful, but it also creates more places where permissions, memory and tool invocation can drift from the original intent. For governance, the key shift is that execution becomes stateful and therefore harder to reason about with static reviews alone.

Practical implication: policy and approval logic need to follow the graph, not sit only in the application wrapper.

LangSmith as tracing, evaluation and production visibility

LangSmith sits above the runtime and records runs, nested calls, traces and evaluation data. That gives teams visibility into what an agent actually did, including LLM calls, tool calls and subgraph activity. Visibility, however, is not the same as control. A trace tree can show that an agent called a tool, but it does not by itself decide whether that tool call should have been allowed or whether the underlying credential scope was appropriate. Observability is essential, but it is downstream of authorisation.

Practical implication: use tracing to investigate and improve agent behaviour, but pair it with pre-execution access controls and runtime policy enforcement.

Threat narrative

Attacker objective: The objective is to exploit overly broad agent permissions and execution pathways to gain access, exfiltrate data or manipulate downstream systems.

1. Entry occurs when an AI agent is given access to tools, models or data sources through a framework stack that was designed for composition rather than governance.
2. Credential access or tool abuse follows when the agent can call external systems with standing permissions that are broader than the task requires.
3. Escalation happens when stateful loops or branching logic let the agent repeat actions, chain tools or widen the blast radius of a mistake.
4. Impact is loss of control over what the agent can access, trace or execute, which can expose sensitive data or trigger unintended downstream actions.

Breaches seen in the wild

• Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.
• LiteLLM MCP auth bypass 2026: An exploited LiteLLM MCP auth bypass and default sk-1234 master keys let attackers steal AI gateway master and provider API keys.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Agent frameworks create a governance stack, not just a software stack. The article makes clear that LangChain, LangGraph and LangSmith solve different layers of the same system, which is exactly why control failure now happens across composition, orchestration and observability. Security teams should stop reviewing agent projects as though they were single applications and start treating them as multi-layer control planes. The practitioner conclusion is that governance must be layered as deliberately as the framework stack itself.

Stateful orchestration changes the identity problem inside agents. Once an agent can loop, branch and checkpoint, the question is no longer only what the model can generate, but what the agent is entitled to do at each step. That makes agent authority closer to non-human identity governance than to ordinary application runtime management, especially where tool use is delegated and persistent. The practitioner conclusion is that access scope must be evaluated per action path, not just per application.

Observability without authorisation is detection after the fact. LangSmith-style tracing helps teams see what happened, but it does not replace policy enforcement at issuance or invocation time. This is a classic control gap in emerging AI stacks: teams add telemetry before they add bounded privilege. The practitioner conclusion is that traceability should support investigation and tuning, while allowlists, approvals and scoped credentials handle prevention.

Authorization propagation is the named gap in modern agent design. When a graph engine can call tools through nested steps, the original caller context often fails to propagate into each downstream action with enough precision. That creates a boundary problem between user intent, agent plan and tool execution that conventional app reviews miss. The practitioner conclusion is to treat propagated privilege as a first-class design requirement, not an implementation detail.

Production AI governance now depends on control placement, not framework preference. The practical decision is where to enforce policy, not which framework is most fashionable. Teams that rely on the framework alone will get better orchestration and better traces, but not necessarily better accountability or least privilege. The practitioner conclusion is to map each control to the layer that can actually stop misuse.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Authorisation Guide

What this signals

Authorization propagation is the real design problem in agent stacks. Once execution can branch, loop and call tools, every downstream action needs an explicit entitlement boundary rather than inherited trust. That makes agent governance more similar to NHI control than to traditional app review, because authority must be checked where the action is taken.

Telemetry should be treated as an after-action control, not the control itself. Traces help teams reconstruct what happened, but they do not prevent a tool call, revoke a credential or constrain a prompt-driven action path. The practical shift is to move policy earlier in the lifecycle, before the agent can compound a mistake.

AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026. That pattern reinforces the same lesson here: the agent layer is now surrounded by infrastructure that often leaks faster than the model itself, so governance has to cover the whole stack.

For practitioners

• Define control points for each agent layer Separate build-time composition, runtime orchestration and post-execution tracing into distinct governance responsibilities so policy does not blur across the stack.
• Scope tool permissions per agent step Treat each tool call as a governed action with its own entitlement boundary, especially where a stateful graph can branch or repeat execution.
• Instrument agent traces for incident review Capture run trees, nested calls and checkpoint transitions so you can reconstruct agent behaviour and investigate unexpected tool use quickly.
• Review delegated access before production rollout Validate what secrets, APIs and external systems the agent can reach before enabling multi-step workflows in shared environments.

Key takeaways

• LangChain, LangGraph and LangSmith solve different layers of the agent stack, but the governance problem emerges when teams assume composition, orchestration and tracing are equivalent to control.
• Stateful agents expand the control surface because branching, looping and checkpointing create more opportunities for privilege drift and unexpected tool use.
• The decisive security question is where authorisation lives, because observability and execution traces cannot stop an over-permissioned agent from acting.

Key terms

• Agent Stack: The layered set of tools used to build, orchestrate and observe AI agents. In this article, the stack includes application composition, stateful execution and tracing. Governance has to follow those layers because each one creates a different control problem.
• Stateful Workflow Orchestration: Stateful workflow orchestration is the design of AI processes that remember prior steps and use that context to guide later actions. It matters when outcomes depend on history, such as long conversations, research flows, or troubleshooting paths. The state can improve consistency, recovery, and auditability across complex execution paths.
• Run Tree: A run tree is a nested trace structure that records parent and child operations across a model workflow. It is useful for debugging, evaluation, and incident review because it shows the sequence of calls, but it does not by itself enforce access or stop unsafe behaviour.
• Authorization Propagation: Authorization propagation is the time it takes for a membership or role change to become effective across the systems that enforce access. In practice, it determines whether the policy state and the runtime state agree. Slow propagation creates stale access windows that identity teams must treat as a governance risk, not a convenience issue.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity and machine identity security for teams building control around delegated access. It gives security and platform practitioners a common language for governing agents, credentials and lifecycle boundaries.