TL;DR: Agent speed becomes durable only when specifications, evaluation harnesses, observability, guardrails, and cost-per-outcome metrics are built into the lifecycle, not added after deployment, according to Arize’s analysis of CVS Health’s AI delivery practices. The implication is that production readiness now depends on governed autonomy, not model quality alone.
At a glance
What this is: Arize argues that AI agent production readiness depends on evaluation-driven development, not just stronger models, because specs, tests, observability, and guardrails are what make speed repeatable.
Why it matters: For IAM and NHI practitioners, this matters because autonomous systems need governed identity, scoped permissions, and measurable controls before they are allowed to act in production.
Context
Enterprise AI often fails at the handoff from impressive prototype to dependable production system. The core gap is governance: organisations can demo an agent quickly, but they cannot safely scale it without specifications, evaluation criteria, monitoring, access controls, and cost discipline. Where AI systems are making decisions or taking actions, those controls start to look very similar to identity and privilege governance for non-human actors.
This article uses CVS Health as an example of how delivery speed becomes durable only when the operating model around the agent is disciplined. The identity angle is real here: once an AI system can trigger code changes, workflows, or operational actions, it needs clearer authorisation boundaries than a conventional development tool. The starting position described here is ambitious but not unusual for teams trying to move beyond pilot purgatory.
Key questions
Q: How should security teams limit the risk from AI agents that have access to production systems?
A: Security teams should scope every agent to the smallest set of actions and resources needed for its task, then remove standing privilege wherever possible. Use short-lived credentials, explicit approval for sensitive actions, and continuous review of what each identity can reach. The goal is to make compromise hard to turn into lateral movement or data exfiltration.
Q: Why do AI systems need governance beyond model performance metrics?
A: Model accuracy alone does not show whether an AI system is lawful, explainable, or defensible. Governance is needed because enterprises must prove who owns each system, what data it can access, how decisions are logged, and what controls exist for privacy and accountability. Without that layer, a well-performing model can still create compliance, security, and legal exposure.
Q: What do security teams get wrong about AI model evaluation?
A: They often collapse quality into a single score and ignore output format, refusals, and latency. In real workflows, those factors determine whether the result can be consumed safely by downstream systems. Good governance treats evaluation as an operational control, with separate thresholds for correctness, completion, and timing.
Q: How do organisations decide whether an AI agent is ready for broader access?
A: They should look for evidence that the agent can meet its business metric, sustain quality across drift, and operate within scoped permissions without creating excessive review or remediation work. If the workflow cannot justify its outcome cost, broader access is premature.
Technical breakdown
Why pilot purgatory appears in AI delivery
Pilot purgatory appears when an AI system can demonstrate usefulness faster than the organisation can define how to evaluate, govern, and operationalise it. A polished demo proves capability in a narrow moment, but production needs release criteria, monitoring, ownership, rollback, and evidence that the output remains correct under changing data and usage. Evaluation-driven development closes that gap by treating specs, tests, and production traces as part of the product, not afterthoughts. The practical issue is not whether the model can generate output. It is whether the surrounding system can decide when that output is acceptable, when it is risky, and when it must stop.
Practical implication: build the evaluation and governance layer before expanding agent autonomy.
How specifications and dependency maps control agent behaviour
Specifications give both humans and agents a shared source of truth for intended behaviour, edge cases, and acceptance criteria. That matters because AI can generate code or workflows that compile yet still misunderstand the business requirement. Dependency maps add the next layer by showing what a change might break across services, data flows, and shared components. In identity terms, this is a form of authorisation planning for non-human actors: the system must know what the agent is allowed to touch and what blast radius a change could create. Without that context, speed simply moves risk earlier in the lifecycle.
Practical implication: pair every agentic change with a clear spec and a dependency map that bounds its blast radius.
What evals and observability do that demos cannot
Evaluation harnesses turn agent quality into something measurable. Golden datasets, regression suites, traces, and continuous evaluation let teams see whether an agent still behaves correctly after model updates, data drift, or changes in user behaviour. Observability is especially important because an apparently reasonable output can hide a bad tool call, a flawed decision path, or an unsafe side effect. For production AI, the question is no longer whether the agent looked right in a demo. It is whether the system can prove that it remains right under real conditions, and whether it can surface enough evidence to justify continued autonomy.
Practical implication: require traces, regressions, and behavioural thresholds before allowing production actions.
NHI Mgmt Group analysis
Evaluation-driven development is becoming the governing model for AI autonomy. The article shows that speed only becomes safe when a system can measure quality, cost, and failure in production, not just at demo time. That is a governance shift as much as an engineering shift because release decisions now depend on evidence across the lifecycle. For identity teams, the lesson is that autonomous systems need control points at issuance, execution, and rollback, not only at build time.
AI agents are forcing security teams to treat access as a runtime decision. Once an agent can generate code, trigger workflows, or take operational action, its permissions matter as much as the model itself. This is where NHI governance intersects with agentic AI security: authorisation, traceability, and scoped action boundaries become the real control plane. Practitioners should treat the agent as a non-human actor whose privileges must be designed, reviewed, and continuously monitored.
Cost-per-outcome is the missing discipline in many AI programmes. Token counts do not capture retries, human review, escalation, or remediation, which means a cheap model can still create an expensive operating system. That makes economic measurement part of security governance because unmanaged cost pressure often leads teams to widen autonomy before controls are mature. The practical conclusion is simple: if the workflow cannot justify its outcome cost, it is not ready for broader trust.
Specifications now function as governance artefacts, not just engineering notes. The article makes clear that the spec is where intent, acceptance, and review criteria become visible to both humans and machines. In agentic environments, that document is part of control design because it constrains what the system may do and how success is judged. Teams should therefore manage specifications with the same seriousness they apply to privileged access policy and change control.
From our research library:
- 53% of security leaders expect AI to run major portions of their infrastructure autonomously within the next three years, according to the 2026 Infrastructure Identity Survey.
- 19% of organisations give AI systems dramatically more access than human employees, nearly one in five granting unrestricted privilege, according to the 2026 Infrastructure Identity Survey.
- Read next: AI Agent Observability, Audit and Incident Response Guide
What this signals
Evaluation is becoming the control plane for agentic AI. As organisations move from experimentation to production, the question is no longer whether an agent can act, but whether its actions are measurable, reversible, and tied to a business outcome. For practitioners, that means release governance must include scoped permissions, traces, and rollback logic before autonomy widens.
AI governance debt accumulates when teams defer specs and test harnesses. Every pilot that skips structured evaluation creates a larger gap between what the system appears to do and what it will do under real load. That gap is where both operational failure and identity abuse become more likely, especially when non-human actors are granted broad access without lifecycle controls.
53% of security leaders expect AI to run major portions of their infrastructure autonomously within the next three years, according to the 2026 Infrastructure Identity Survey. That expectation makes evaluation-driven development a present-day control requirement, not a future concern.
For practitioners
- Define release criteria before autonomy expands Write acceptance criteria, failure thresholds, and rollback conditions before agents are allowed to touch production workflows or code paths.
- Treat specifications as control artefacts Make the specification the shared source of truth for intent, edge cases, and business rules so humans and agents are judged against the same standard.
- Build evaluation harnesses around real business cases Create golden datasets and regression suites from the outcomes that matter, then rerun them whenever prompts, models, or dependencies change.
- Scope agent permissions to the task Grant only the access needed for a specific workflow and review which actions the agent can take without additional approval.
- Measure cost per outcome, not token spend Track the full cost of retries, review, exceptions, and rework against the business KPI the agent is meant to move.
Key takeaways
- AI agent programmes stall when production controls are treated as add-ons instead of design inputs.
- Evaluation harnesses, observability, and scoped access turn agent speed into durable delivery.
- Without cost-per-outcome measurement, teams can confuse cheap inference with operational efficiency.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and MITRE ATT&CK address the attack and risk surface, while NIST AI RMF sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Agent autonomy here depends on tightly scoped identity and privilege boundaries. |
| ASI08 — Cascading Failures | The article stresses dependency maps and blast-radius control to prevent agent changes from spreading. | |
| Recommendation — Apply ASI03 by constraining agent privileges to the minimum needed for each workflow and reviewing escalation paths. Use ASI08 to require dependency awareness and containment before permitting agentic changes in production. | ||
| NIST AI RMF | GOVERN — AI Governance and Accountability | The article is fundamentally about governing AI behaviour through specs, evaluations, and review. |
| MANAGE — AI Risk Treatment and Response | Production readiness depends on monitors, rollback, and intervention loops when agents drift or fail. | |
| Recommendation — Use GOVERN to define accountability, release thresholds, and oversight for agent autonomy. Use MANAGE to create rollback, fallback, and human-review triggers for agentic workflows. | ||
| MITRE ATT&CK | TA0006 — Credential Access | The article's access control concerns overlap with how non-human actors obtain and use privileges. |
| Recommendation — Map agent permission sprawl to TA0006 and reduce standing access that exceeds the task scope. | ||
Key terms
- Evaluation Driven Development: A development approach that treats measurement as a first-class part of building AI systems. Teams define test suites, trace runs, and compare outcomes continuously so they can prove whether a change improved quality or introduced regressions.
- Cost per outcome: A measure of the full cost required to complete a business task, including retries, review, exceptions, and rework. It is more useful than token cost alone because it reflects whether an AI workflow is actually efficient at scale.
- Golden Dataset: A curated set of expected inputs and outputs used to test whether an AI system behaves consistently across changes. In governance terms, it becomes evidence that model or workflow changes did not introduce regressions that would affect reliability, safety, or access-related behaviour.
- Agentic Guardrail: A guardrail layer that can use tools and code context to make better security decisions for autonomous coding agents. Instead of reviewing output in isolation, it gathers surrounding information to judge whether code is vulnerable or malicious. This improves detection quality when risks are distributed across a larger codebase.
Deepen your knowledge
The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and machine identity security. It helps practitioners build the control thinking needed to govern non-human actors safely across modern programmes.
Published by the NHIMG editorial team on August 18, 2026.
Updated on October 12, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org