TL;DR: Agentic AI breaks the deterministic testing assumptions enterprise teams used for traditional software, and Fiddler argues that observability, auditability, hybrid architectures and ROI-first use case selection are now the practical foundation for safe deployment. The real governance shift is that agent behaviour must be treated as a managed system property, not a post-launch surprise.
Editorial analysis by NHI Mgmt Group, based on content published by Fiddler: “Beyond Predictability: Lessons Learned from Building Agentic Systems”.
Key questions
Q: How should security teams govern agentic AI as it moves into production?
A: Security teams should govern agentic AI as a class of non-human identity, not as a generic application feature.
Q: Why do agentic AI systems increase risk for API security and governance?
A: Agentic AI increases risk because it can interpret context, make decisions, and execute multi-step actions without waiting for direct human approval.
Q: What are the signs that AI agent observability is failing?
A: The main signs are missing tool metadata, disconnected traces between agent hops, unexplained retries, and an inability to reconcile a successful call with the actual state change it caused.
Practitioner guidance
- Define agent observability requirements before deployment Log tool calls, prompt changes, token usage, human interventions and execution outcomes so every agent decision can be reconstructed.
- Classify agent use cases by business criticality Prioritise only the workflows where automation creates measurable value and keep experimental deployments out of high-risk processes until governance is mature.
- Add validation layers to high-impact decisions Use deterministic checks, knowledge grounding or human approval for outputs that influence access, financial actions or regulated decisions.
Bottom line: Agentic AI cannot be governed safely with deterministic testing alone because behaviour varies across executions and tool paths.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
Agentic AI governance now starts at execution time, not review time. The article’s core lesson is that unpredictability is a system property, not an exception. That means controls built around static approval, after-the-fact testing or periodic review will miss the moment when an agent actually acts. For identity teams, this is where agent behaviour begins to resemble non-human identity governance: what matters is not just who built the workflow, but what authority it exercised during execution.
A few things that frame the scale:
- A Harris Poll survey of more than 300 technology decision-makers found that 86% expect agentic AI to deliver positive ROI, yet fewer than half had AI governance policies in place.
A question worth separating out:
Q: What should organisations do when AI adoption outpaces governance?
A: Slow the rollout of new use cases until the identity model is clear, the permissions are bounded, and the lifecycle process is in place. Security leaders should fund AI only where it supports existing cybersecurity objectives and where access can be monitored, rotated, and removed on time.
👉 Read our full editorial: Agentic AI systems need observability, auditability and ROI discipline