Join our Newsletter — 33% off our NHI Course

Is 95% step reliability enough for a 20-step agent workflow?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 2 years ago
Posts: 20217
Topic starter  

TL;DR: 36% success on a 20-step task shows how fast a 95% per-step rate can decay, in Agentic Fabriq's case work. Long agent workflows need narrower scope and repeated-task checks, not a single pass score.

Editorial analysis by NHI Mgmt Group, based on content published by Agentic Fabriq: “Why Your Agent Works in the Demo and Fails in Production”.

Key questions

Q: Why do multi-step agent workflows fail even when every step seems acceptable?

A: Because each step multiplies the chance of failure.

Q: What signs show that an agent is failing silently rather than obviously?

A: Look for duplicated writes, status updates that do not match the underlying state, partial completions reported as done, and results that vary across repeated runs.

Q: How do security teams know if an agent workflow is actually reliable?

A: They measure it with repeatable evaluations, not with launch-time confidence.

Practitioner guidance

  • Model end-to-end success, not just step success Count the steps in each agent workflow and estimate the full-task success rate from per-step reliability before you move into production.
  • Instrument per-step outcomes and variance Log each intermediate decision, tool call, write, and verification result so you can see where reliability decays.
  • Add external verification for state-changing actions Require schema validation, reconciliation, and idempotent writes for any agent action that changes records, permissions, or financial state.

Bottom line: Long agent workflows can fail even when each individual step looks acceptable, because the end-to-end result is the product of all those steps.

Glossary terms: Compounding Error, Pass K, Silent Failure, Reliability Horizon

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →


This topic was modified 7 days ago by NHI Mgmt Group

   
Quote
(@mr-nhi)
Member Moderator
Joined: 6 months ago
Posts: 20707
 

Compounding error is the central governance problem in agentic workflows: teams often evaluate each step in isolation and miss that the final outcome is the product of all steps. That makes a 95% score per step look acceptable while the full process is operationally brittle. For identity-heavy workflows such as approvals, provisioning, and exception handling, the failure mode is not dramatic breakage but quiet drift into incorrect state. Practitioners should govern the full task path, not the model in isolation.

A question worth separating out:

Q: Should organisations narrow agent scope or keep adding retries and guardrails?

A: Narrowing scope usually wins when the workflow is long or brittle. Retries help with transient faults, but they do little against stale context, repeated reasoning errors, or bad state propagation. If the task only works when everything goes right, it is too broad. Better governance comes from fewer autonomous steps, clearer exit criteria, and human escalation where the risk of compounding error is high.

👉 Read our full editorial: Why agent reliability collapses as task length grows


This post was modified 7 days ago by NHI Mgmt Group

   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.