Agentic AI workflows need stronger safety, accuracy and governance

At a glance

What this is: Fiddler argues that agentic AI workflows increase enterprise automation value, but also widen risk around prompt injection, misinformation, hallucination and unsafe data access.

Why it matters: For IAM, NHI and AI governance teams, the key issue is that agentic systems need runtime authorization, monitoring and intervention controls, not just model-quality checks.

Context

Agentic AI changes the security question from whether a model is accurate to whether a workflow can act safely while it is running. Once an AI system is allowed to plan, call tools and chain decisions inside business processes, static review checkpoints stop being enough on their own.

That creates a governance gap that sits between AI safety and identity control. The article’s core message is that organisations need to treat agentic workflows as governed execution environments, with explicit access controls, monitoring and human intervention paths when outputs or behaviour drift outside acceptable bounds.

Key questions

Q: What breaks when runtime guardrails are missing from agentic AI?

A: Without runtime guardrails, the organisation loses control at the moment the agent takes a real-world action. Prompt injection, tool misuse, overbroad permissions, and non-deterministic behaviour can all result in data leakage, unauthorized transactions, or destructive changes before anyone can intervene.

Q: Why do agentic AI systems need more than model accuracy metrics?

A: Model accuracy metrics can miss the risk that matters most in agentic systems. An agent may produce a plausible answer and still make the wrong tool call, leak context or trigger a harmful workflow step. Teams need combined monitoring for safety, faithfulness and business outcomes, because the control objective is safe execution, not just a better score.

Q: What are the signs that an agentic workflow is overexposed?

A: Common signs include broad tool access, reused credentials across workflows, unclear human approval boundaries, and agents that can move from a prompt to a business action without policy checks. If the workflow can still function after removing obvious safeguards, the system is probably too permissive and too hard to govern safely.

Q: How should organisations govern AI agents alongside human identity and device access?

A: Organisations should treat AI agents as a separate identity class with their own entitlement boundaries, logging expectations, and approval model. Human IAM controls often assume interactive sign-in and review cycles, which do not fit autonomous or programmatic access. The safer approach is to define actor-specific policy and verify which access paths can be delegated without expanding trust unnecessarily.

Technical breakdown

Why agentic workflows change the control model

Agentic AI is not just a chat interface with more steps. It is a workflow where the model can decide what to do next, often by selecting tools, calling APIs, or passing context into later steps. That autonomy creates a larger attack surface because the model is no longer only producing text. It can also trigger actions, leak context, or chain a weak output into a harmful downstream decision. Traditional model evaluation focuses on output quality, but agentic risk also depends on tool access, session context, and whether the workflow can be steered by adversarial input.

Practical implication: Design controls around action boundaries, not only output validation.

Why observability has to cover safety, accuracy and behaviour

The article points to a core monitoring problem: generative outputs do not have stable ground truth in the way many classical ML systems do. That means teams cannot rely on a single accuracy metric to tell them whether an agent is operating safely. Instead, monitoring has to combine LLM-specific signals such as faithfulness, hallucination and safety with business outcomes and workflow-level behaviour. This is especially important when an agent touches customer data, regulated workflows, or delegated access paths, because small errors can become control failures very quickly.

Practical implication: Measure both model quality and workflow outcomes in the same monitoring layer.

How identity and authorisation become part of agent safety

Agentic systems are not governed well by content filters alone because many failures happen through over-broad access, not just unsafe text. When an agent can authenticate to tools, retrieve sensitive data, or pass context between systems, the real question becomes what it is allowed to do at each step. That is where identity governance intersects with AI governance. The article’s emphasis on authentication, authorisation and secure transmission reflects a broader principle: agent safety depends on constraining the identity and privilege of the system doing the work, not just screening its prompts.

Practical implication: Bind each agent to least-privilege access and explicit step-level permissions.

Threat narrative

Attacker objective: The attacker aims to make the agent execute unintended actions or reveal sensitive information by exploiting its delegated autonomy.

1. Entry occurs when an adversary supplies adversarial prompts, misleading context or poisoned inputs to an agentic workflow that can call tools or chain decisions.
2. Credential or authority abuse follows when the workflow uses its delegated permissions, allowing the malicious input to influence actions beyond the intended user request.
3. Escalation happens when the agent propagates the bad instruction across later steps, APIs or connected systems, turning one compromised interaction into a broader workflow failure.
4. Impact is unsafe execution, data exposure, inaccurate business decisions or reputational and financial damage caused by the agent acting on manipulated context.

Breaches seen in the wild

• McKinsey AI platform breach: McKinsey AI platform hack exposed 46M chats and sensitive data.
• Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Agentic AI governance debt is now a live security issue: organisations are scaling autonomous workflows faster than they are building the control layers those workflows require. The article shows that model quality alone does not govern action-taking systems because tool use, delegation and context flow create new failure modes. Practitioners should treat this as a governance backlog, not a future risk.

The real control boundary sits at runtime, not at model release: once an agent can select tools, move context and trigger business processes, post-training safety checks are too late to catch every failure. That makes runtime observability, policy enforcement and intervention paths the decisive controls for AI governance. Teams should assume that static approval gates will miss the most important agent failures.

Identity is becoming the operating layer for agent safety: an agent that can authenticate, inherit permissions or reuse a session needs the same scrutiny as any other non-human identity. The governance problem is not just what the model says, but what the system is authorised to do. That shifts security teams toward per-action authorisation, scoped credentials and explicit lifecycle control for agent access.

Ground-truth collapse: agentic systems break the assumption that one metric can certify performance because outputs are contextual, probabilistic and sometimes non-deterministic. The article’s call for composite evaluation is effectively a warning that conventional ML scorecards do not map cleanly to business risk. Practitioners should align safety measurement with actual workflow outcomes, not model vanity metrics.

Agentic AI safety and NHI governance are converging: as workflows gain tool access and delegated authority, they start to resemble other non-human identities that must be inventoried, authorised and monitored. This is where AI security, IAM and PAM overlap in practice. Security programmes should manage agents as governed digital actors with explicit rights, boundaries and revocation paths.

From our research library:

• According to PwC, 79% of organizations have already adopted AI agents to some degree.
• Read next: Agentic AI Identity Maturity Model

What this signals

Agentic AI adoption is already colliding with weak credential hygiene: 67% of organisations still rely heavily on static credentials despite the risks they pose to agentic AI deployments. That matters because autonomous workflows inherit the weaknesses of the identities they use, so access control has to move from static secrets to governed runtime authority.

Identity governance becomes the control plane for agentic safety: once an AI workflow can call tools, reuse sessions and pass context between systems, the primary question is not model quality but who or what is authorised to act. Teams should expect procurement, security architecture and AI governance to converge on the same runtime control decisions.

AI governance debt will surface as operational debt: the more organisations expand agentic workflows without a matching authorization model, the harder it becomes to explain, audit or safely revoke what those systems can do. That is why agent identity, credential scope and human intervention paths need to be designed together from the start.

For practitioners

• Define step-level agent permissions Map each agent workflow to the exact data sources, tools and actions it needs, then remove any inherited access that is not required for the task.
• Add runtime guardrails for tool use Enforce policy checks before tool calls, context hand-offs and external actions so unsafe prompts cannot become executed instructions.
• Replace single-score evaluation with composite monitoring Track faithfulness, safety, hallucination rates and business outcome signals together so monitoring reflects workflow risk instead of a single ML metric.
• Build pause and kill-switch paths Provide human intervention for high-stakes workflows so suspicious agent behaviour can be stopped before it reaches downstream systems.
• Treat agent credentials as governed identities Inventory each agent credential, tie it to an owner and rotate or revoke it when the workflow, model or business use case changes.

Key takeaways

• Agentic AI expands enterprise automation, but the real security challenge is governing what the system can do while it is acting, not only judging what it says.
• Monitoring must combine workflow behaviour, safety signals and business outcomes because a single accuracy metric cannot capture agent risk.
• Agent safety depends on identity and authorization controls that bind each workflow to scoped access, revocation and human intervention paths.

Key terms

• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions, including calling APIs, writing code, and orchestrating other agents, with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.
• Runtime Guardrail: A control applied while an AI agent is operating, not just during configuration or review. Guardrails can block dangerous tool calls, require approval for sensitive actions, or stop data leakage before it reaches systems or users.
• Human-In-The-Loop Evaluation: A human-in-the-loop evaluation is a review process where subject matter experts score LLM outputs against a defined rubric. It is used when automated scorers cannot reliably judge factual accuracy, policy compliance, tone, or domain nuance, especially in regulated or high-stakes environments.
• Composite Metric: A composite metric combines multiple measures into one view of system performance. For agentic AI, it should blend safety, faithfulness, task success, and business outcome signals so teams can judge whether the workflow is both effective and controlled.

Deepen your knowledge

The NHI Foundation Level course covers NHI governance, agentic AI identity, machine identity security and secrets management through an accredited practitioner programme. It helps security teams connect identity control to autonomous workflow risk without losing operational clarity.