LangChain, LangGraph and LangSmith: the governance gap in agent stacks

At a glance

What this is: This is a comparison of three AI framework layers and its key finding is that mature agent systems require building, orchestration, and observability to be governed separately.

Why it matters: IAM, NHI, and AI governance teams need to understand where execution logic ends and tool access, tracing, and accountability begin, because agent stacks introduce control points that traditional application reviews do not cover.

👉 Read TruFoundry's comparison of LangChain, LangGraph, and LangSmith

Context

LangChain, LangGraph, and LangSmith solve different layers of the AI application stack, but that separation also creates a governance problem. One layer builds model workflows, another controls multi-step agent execution, and the third records what happened after the fact. For identity and access teams, the important question is not which framework is better, but where tool access, state, and auditability are enforced when AI systems begin to act like operational actors.

This matters because agentic systems are increasingly treated as production software rather than experiments, yet they often inherit weak assumptions about who or what can call tools, hold state, or trigger downstream actions. Where those systems connect to MCP, external APIs, or sensitive data sources, the control model starts to look less like simple application design and more like governance of non-human identity-like runtime behaviour.

Key questions

Q: How should security teams govern agentic AI as it moves into production?

A: Security teams should govern agentic AI as a class of non-human identity, not as a generic application feature. That means assigning ownership, scoping permissions tightly, logging every tool action, and revoking access on a defined lifecycle. Production rollout should require clear approval points for high-risk actions and continuous monitoring for drift.

Q: Why do AI agents create a governance problem for IAM teams?

A: AI agents create a governance problem because they authenticate and act as autonomous software entities with tool access. If their actions are logged only as application activity, teams lose accountability, context, and revocation clarity. IAM must therefore extend to agent identity, delegated authority, and control-plane audit trails.

Q: What breaks when tracing exists without enforcement for AI agents?

A: You get visibility into failures without a mechanism to prevent them. Traces can show a bad tool call, a sensitive data access, or an unsafe branch, but they do not stop the next one. That leaves teams with better forensics but the same operational exposure.

Q: Should teams use LangChain, LangGraph, and LangSmith together or separately?

A: Use them based on the system’s complexity, but govern the combined stack as one operational unit once agents start branching, persisting state, or calling tools with business impact. At that point, the question is not which framework to pick, but where control, audit, and accountability live.

Technical breakdown

Runnable pipelines vs stateful graphs

LangChain’s core abstraction is the Runnable pipeline, where prompts, models, retrievers, and parsers can be composed into a linear sequence. That works well for deterministic workflows such as retrieval-augmented generation and structured extraction. LangGraph adds a different execution model: a stateful graph with nodes, edges, conditional routing, and checkpoints. The key distinction is that the agent can revisit decisions, branch based on intermediate results, and persist state across steps instead of only passing output forward once.

Practical implication: teams need to decide whether a workflow is linear enough for chain review or stateful enough to require explicit agent governance.

Tracing, run trees, and evaluation data

LangSmith sits above execution and records inputs, outputs, child calls, and nested tool activity as a run tree. That trace structure is what makes debugging possible when an AI system fails in a multi-step path. It also introduces evaluation workflow through datasets, custom evaluators, LLM-as-judge scoring, and human review. In governance terms, tracing is evidence, not control. It tells you what the system did, but it does not itself restrict what the system may do.

Practical implication: observability must be paired with policy controls for tools, data, and state before traces can support accountable operations.

Tool access and agent control boundaries

The source article explicitly notes that tracing does not manage credentials or model-level governance, which is the more important lesson for production use. Once an agent can call tools, access data, or route through external systems, the true control point moves to authorization at the gateway or tool boundary. In identity terms, that means the agent stack behaves like a governed runtime with delegated access, not just a software library. The risk is uncontrolled expansion of effective privilege as agent capability increases.

Practical implication: bind agent actions to least-privilege tool permissions and separate runtime observation from authorization enforcement.

NHI Mgmt Group analysis

LangGraph-style agents are becoming runtime identities, not just application code. Once a system can branch, loop, hold state, and invoke tools, it begins to behave like a non-human actor with persistent operational effects. That changes the governance question from code quality to controlled delegation. Teams should treat agent execution paths as identity-bearing workflows that need explicit scope, auditability, and revocation logic.

Observability without enforcement creates an evidence trail, not a safety boundary. LangSmith-style tracing is valuable for debugging and post-incident review, but it does not stop an agent from calling the wrong tool or accessing the wrong data. This is the same governance mistake seen in other identity domains: logging is not authorization. Practitioners should align tracing with tool-level policy enforcement and access boundaries.

Stateful orchestration widens the attack and error surface compared with linear chains. A graph that can checkpoint, branch, and retry is operationally powerful, but it also creates more opportunities for policy drift, prompt injection effects, and unintended tool invocation. The right control model is not to avoid orchestration, but to define where state may persist and where execution may be interrupted. Teams should map these flows to NIST AI RMF and OWASP Agentic AI Top 10 expectations.

Named concept: agent control-plane drift. As teams adopt separate tools for building, orchestrating, tracing, and hosting AI agents, governance responsibilities fragment across components that were not designed as one control system. That fragmentation makes it easier for access policy, audit policy, and runtime policy to diverge. Practitioners should treat the stack as one governed plane, not three independent products.

What this signals

Agent control-plane drift: once building, orchestration, tracing, and hosting are split across different components, policy can fragment faster than the AI programme matures. The programme response is to define a single control boundary for tools, state, and audit before the agent estate expands further.

The practical signal is that trace visibility will not solve delegated access problems on its own. Teams should align gateway policy, runtime policy, and review policy to the same operational boundary, then test that boundary against real tool calls and sensitive data paths.

For practitioners

• Define the agent execution boundary Document which workflows remain linear, which require stateful orchestration, and which must be treated as governed agent loops rather than simple chains.
• Separate tracing from authorization Use LangSmith or similar telemetry to capture runs, but enforce tool permissions, data access, and environment constraints at the gateway boundary.
• Classify agent tools by privilege level Map each tool, API, and data source to the minimum access it needs, then review any agent that can reach sensitive systems or secrets.
• Add checkpoint and interrupt rules Require explicit pause points for high-impact actions such as external writes, credential use, or multi-step side effects that could escape normal review.

Key takeaways

• LangChain, LangGraph, and LangSmith are different control layers, not interchangeable tools.
• AI agent programmes now need both observability and enforcement because tracing alone cannot constrain delegated runtime behaviour.
• The governance challenge is no longer just model development, but the control of tool access, state, and accountability across the whole agent stack.

Key terms

• Runnable Pipeline: A Runnable pipeline is a linear composition model where each step can be chained into the next and treated as one callable unit. It suits deterministic LLM workflows such as retrieval, prompting, and structured parsing, but it does not naturally express loops or persistent state.
• StateGraph: A StateGraph is a graph-based execution model for agents that need memory, branching, retries, and checkpoints. Nodes update shared state and conditional edges decide the next step, which makes the workflow more expressive but also harder to govern without clear policy boundaries.
• Run Tree: A run tree is a nested trace structure that records parent and child operations across a model workflow. It is useful for debugging, evaluation, and incident review because it shows the sequence of calls, but it does not by itself enforce access or stop unsafe behaviour.
• Agent Control Plane: An agent control plane is the policy layer that decides what an AI agent may do before execution. It connects discovery, identity, and authorization so the organisation can enforce task-scoped limits instead of relying on static registration or after-the-fact review.

What's in the full article

TruFoundry's full article covers the implementation detail this post intentionally leaves for the source:

• The exact LCEL composition patterns and Runnable interfaces used to build LangChain pipelines.
• The StateGraph mechanics behind LangGraph checkpoints, conditional edges, and shared state.
• The LangSmith run tree, evaluation workflow, and hosting options in more operational detail.
• The article's own feature-by-feature comparison table for selecting a framework based on use case.

👉 The full TruFoundry article covers the framework details, comparison table, and production use cases in more depth.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management for teams building controlled runtime access. It helps practitioners connect identity policy to agent and application workflows that now behave like operational actors.