Context and evaluation now define reliable enterprise AI agents

At a glance

What this is: This is an analysis of why enterprise AI agents fail even when the model is strong, showing that context quality, harness design, and evaluation coverage now shape reliability more than benchmarks alone.

Why it matters: It matters to IAM, NHI, and AI governance teams because agents increasingly act across tools, permissions, and handoffs, turning context drift and authorization design into control issues, not just model-quality issues.

By the numbers:

• Arize says customers run more than 100 million evals every month across their AI systems.
• When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes and as quickly as 9 minutes in some cases.

👉 Read Arize's analysis of why enterprise AI agents still fail despite better models

Context

Enterprise AI reliability is now a governance problem as much as a model problem. The model may reason well, but the surrounding context, permissions, retrieval paths, and handoffs determine whether the agent acts on current, approved business information or on stale assumptions that produce fluent but wrong outcomes.

That distinction matters for IAM and NHI programmes because agents do not just consume data, they exercise access. Once a system can call tools, use memory, and delegate work, the control plane must account for who or what is authorised to act, what context it may rely on, and how those decisions are traced when behaviour drifts. This pattern is becoming normal in enterprise deployments, not an edge case.

Key questions

Q: What breaks when AI systems reuse stale context after an error?

A: The main failure is loss of conversation isolation. Stale context can make unrelated replies look valid, which weakens user trust and can expose material from another session if the platform misbinds cache or parent-message references. The right response is to validate lineage, isolate session data, and test every error branch that can rebuild context.

Q: Why do AI agents increase non-human identity risk?

A: AI agents increase non-human identity risk because they can execute many actions quickly once they inherit a credential or tool permission. That speed expands blast radius, shortens attacker dwell time, and makes weak delegation more dangerous. The remedy is tighter scoping, continuous verification, and strict separation between observation and execution privileges.

Q: How do you know if agent evaluation is actually working?

A: It is working when it explains failures well enough to change the system, not just when it outputs a score. Good evaluation should point to the layer that failed, produce a reproducible trace, and feed confirmed failures into regression tests. If the score does not change engineering decisions, it is not yet operational.

Q: Should organisations treat context engineering as a governance control?

A: Yes. Context engineering governs what an agent is allowed to know, which sources it trusts, and which business definitions it uses before acting. That makes it a control surface for AI governance, data governance, and identity governance together. Without ownership, versioning, and approval paths, context drift becomes a security and reliability risk.

Technical breakdown

Why strong models still fail inside enterprise agent stacks

A model benchmark measures isolated reasoning, but enterprise agents operate inside a stack of context, retrieval, tools, memory, permissions, and handoffs. Failures often appear when the model receives stale, conflicting, or incomplete business context and still produces a plausible answer or action. That is why evaluation has to move from static prompts to production traces. The system may look correct at the final output while the fault actually sits in retrieval, routing, or permission handling. This is a systems problem, not a model-only problem.

Practical implication: evaluate the full agent path, not just the final response.

How the context layer changes what an agent is allowed to know

The context layer is the governed business meaning that tells an agent what data is trusted, what definitions apply, and which workflows are valid. It typically includes lineage, ontology, policies, certification status, and approval paths. When that layer is incomplete or out of date, the agent may reason from an old definition or a deprecated source and still sound authoritative. In identity terms, this is not only about data quality. It is about which knowledge sources an agent is authorised to rely on before it acts.

Practical implication: treat governed context as a control surface, not documentation.

Why harness engineering now looks like runtime authorisation

The harness is the operational layer that lets an agent retrieve data, call tools, execute code, use memory, and hand off work. As those capabilities expand, the number of possible trajectories expands with them, which increases failure modes and also the blast radius of bad authorisation design. If permissions, tool scope, or recovery logic are weak, the agent can move from harmless reasoning into harmful action without a clear boundary. For identity teams, that makes harness design closely related to least privilege and task scoping.

Practical implication: scope tools and permissions as tightly as privileged access.

NHI Mgmt Group analysis

Confident irrelevance is the right name for a growing enterprise AI failure mode. The article shows that a capable model can still act on stale or conflicting business context and produce a fluent but wrong outcome. That is not a hallucination problem alone, it is a control problem caused by weak context governance and incomplete evaluation. Teams should treat this as a governance debt issue, because the risk grows every time the business changes faster than the context layer.

Agent harnesses are becoming identity enforcement points, even when teams do not describe them that way. Once an agent can call tools, use memory, and delegate work, the harness determines what the system may do in practice. That makes permissions, handoffs, and retrieval boundaries part of the effective authorization model. For IAM and PAM leaders, the lesson is that agent reliability cannot be separated from runtime privilege design.

Evaluation now functions as the evidence layer for AI governance. The article’s emphasis on traces, code evaluators, LLM judges, and agent judges reflects a shift from one-time testing to continuous operational proof. That aligns with NIST AI Risk Management Framework thinking: governance has to be measurable in production, not inferred from benchmarks. Practitioners should build evaluation evidence that can explain why an agent acted, not just whether it answered well.

Context governance is a new form of identity governance for machine action. The context layer described here includes definitions, trust signals, and approval paths that shape how an agent behaves. In that sense, it is governance for machine-held business knowledge and the actions derived from it. This is where AI security, data governance, and identity governance intersect most directly, and it is the intersection practitioners should formalise first.

Named concept: context drift becomes operationally dangerous when AI systems act across tools. Definitions, policies, and trusted sources are no longer passive reference material once an agent can execute tasks. If those inputs drift, the resulting action drift can be harder to detect than a simple answer error. Practitioners should therefore manage context with the same lifecycle discipline they apply to credentials, policies, and other governed identity assets.

What this signals

Agentic systems are pushing identity controls closer to runtime, where permissions, memory, and tool access behave like live authorisation decisions rather than static entitlements. That means security teams need to watch for the same pattern they already see in secrets management: control confidence rising faster than actual remediation and validation performance. The relevant shift is from model assurance to operational assurance, and the governance model must catch up.

Context drift: when governed definitions lag behind the business, the agent becomes technically correct and operationally wrong. That makes lineage, certification, and approval status more than metadata. They are the evidence that lets teams decide whether an agent should trust a source before it acts.

For identity programmes, the practical signal is whether evaluation failures are being turned into durable controls. If traces, regression cases, and context approvals do not feed each other, the organisation is only observing AI risk, not reducing it. Teams should use production evidence to tighten tool scope, context ownership, and handoff rules in the same way they would harden privileged workflows.

For practitioners

• Implement trace-level evaluation Instrument every agent path so teams can see retrieval, tool calls, handoffs, and execution outcomes. Use the trace to distinguish context failure from harness failure before a wrong answer becomes a production incident.
• Govern context as a versioned asset Assign owners, versioning, approval steps, and deprecation signals to business definitions, trusted sources, and policies. Treat stale context as a controlled failure condition, not a documentation issue.
• Scope agent permissions to task boundaries Limit tools, memory, and code execution to the minimum required for the specific workflow, then verify those boundaries in production. The closest analogue is privileged access control, not broad application access.
• Build regression suites from confirmed production failures Convert traced failures into reusable test cases that cover the exact context, retrieval, and recovery conditions that caused the issue. That makes the evaluation system improve with actual behaviour rather than curated examples alone.
• Separate model quality from system quality Track whether a failure came from the model, the context layer, or the harness before changing prompts or switching models. This prevents teams from fixing the wrong layer and masking the real governance gap.

Key takeaways

• Agent reliability is now governed by context quality, harness design, and evaluation coverage, not model choice alone.
• AI systems that can retrieve, call tools, and delegate work create runtime authorisation problems that belong in identity governance.
• Production traces are the evidence layer that turns AI governance from a promise into something teams can test and improve.

Key terms

• Contextual layer: An intermediate governance layer that adds visibility and control across systems not fully covered by the primary IGA stack. It matters when organisations need immediate insight into drift, exceptions, and coverage gaps while they work toward a more mature governance architecture.
• Harness Engineering: Harness engineering is the set of systems around a model that make it reliable in production. It includes orchestration, retries, guardrails, observability, evaluation, and fallbacks, all of which determine whether the agent can be trusted at enterprise scale.
• Trace-Level Evaluation: An evaluation method that inspects the full sequence of agent steps rather than only the final answer. It reveals where retrieval, routing, permissions, recovery, or execution diverged from expected behaviour, making failures diagnosable and repeatable.
• Confident Irrelevance: A failure pattern where an AI system produces a fluent and apparently trustworthy result from the wrong context. The model is not necessarily reasoning incorrectly, but it is applying that reasoning to stale, incomplete, or misaligned information that the business should not trust.

What's in the full article

Arize's full analysis covers the operational detail this post intentionally leaves for the source:

• Trace examples showing how trajectory failures emerge across retrieval, tool use, memory, and handoffs
• The evaluator design choices behind code checks, LLM judges, and agent judges
• The operating loop for turning failed runs into regression data and engineering actions
• How the context layer is mined, versioned, approved, and activated across enterprise systems

👉 Arize's full post covers trace evaluation, context governance, and harness failure patterns in more implementation detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, secrets management, and identity lifecycle controls. It gives practitioners a common framework for governing non-human access as AI systems begin to act across tools and permissions.