Agentic AI workflows need stronger safety, accuracy and governance

At a glance

What this is: This is Fiddler's analysis of how agentic AI workflows change enterprise security, with the key finding that safety, accuracy, and governance must be managed at runtime rather than treated as model-only concerns.

Why it matters: It matters because IAM, NHI, and AI governance teams need controls for agent access, authorization, and monitoring before agentic workflows become embedded in business-critical processes.

👉 Read Fiddler's analysis of developing agentic AI workflows with safety and accuracy

Context

Agentic AI changes the control problem because software can now decide actions, invoke tools, and move through workflows with less human gating. That creates a governance gap between what the model can do and what the organisation has actually authorised, especially where agent decisions touch sensitive data, privileged systems, or downstream business processes. For identity and security teams, the first question is no longer only model quality, but who or what is allowed to act on behalf of the organisation.

The article frames the right response as a combination of access controls, guardrails, monitoring, and human oversight. That is the correct direction, but the deeper point is that agentic systems behave like non-human identities in practice, even when the article does not use that label. Once an agent can trigger actions across applications, the governance model must extend beyond prompts and outputs into identity, privilege, and lifecycle control.

Key questions

Q: How should security teams govern agent tool calls in production?

A: Treat each tool invocation as a separate identity decision, not as a one-time permission granted to the agent. The control model should validate current context, the specific tool being called, and whether the action matches the intended task. This is the only way to keep delegated agent activity auditable and bounded.

Q: Why do agentic AI systems create more security risk than standard chatbots?

A: Agentic systems can turn model output into action, which means a bad instruction can affect code flow, tool use, and downstream state. Standard chatbots usually stop at text generation. Agentic AI adds execution authority, so compromise can lead to disclosure, unauthorized changes, or misuse of credentials.

Q: What do security teams get wrong about prompt guardrails?

A: Teams often treat prompt guardrails as if they were authorisation controls, but they are only one layer of defence. A model that filters unsafe language can still execute hidden instructions inside legitimate content if tool permissions are broad. Guardrails reduce exposure, but they do not replace separate approval checks for sensitive actions.

Q: How do organisations know if agent security controls are actually working?

A: Look for evidence that the platform can inspect traces, classify risky actions, and stop unsafe tool use before completion. Effective controls leave an audit trail that shows why the action was allowed or denied, and they reduce false positives enough that teams can trust them in production.

Technical breakdown

Why agentic workflows create a runtime authorisation problem

Agentic workflows differ from ordinary LLM use because the system can choose actions, sequence tools, and continue execution without a human approving each step. That creates a runtime authorisation problem: the organisation must govern not just what the model says, but what it is allowed to do in the moment. In practice, the risky boundary is between model output and tool invocation, especially when agents can read data, call APIs, or write to business systems. Authentication alone is insufficient if the agent's effective permissions are broader than the task requires.

Practical implication: treat each agent workflow as a scoped identity and authorize every tool call against task-bound policy.

How guardrails and adversarial testing reduce prompt injection risk

Guardrails are runtime controls that intercept unsafe prompts or outputs before they reach the model, the tool layer, or the user. They matter because prompt injection and jailbreaks are not just content problems, they are control bypass problems that try to redirect the agent into unintended behaviour. Adversarial testing should simulate malformed instructions, hidden prompts, retrieval poisoning, and unsafe tool requests so teams can see where the workflow breaks. The goal is not to eliminate all risk, but to define where the system must stop, ask, or refuse.

Practical implication: test the full agent path, including retrieval and tool use, for instruction hijacking before production rollout.

Why LLMOps needs composite metrics, not model scores alone

Traditional ML metrics such as precision and recall do not describe whether an agent is behaving safely across a workflow. Agentic systems need composite measures that combine faithfulness, safety, task success, and business-specific outcomes, because a technically accurate answer can still be operationally unsafe. Observability must cover the entire chain from input to tool use to final action. Human-in-the-loop review remains essential for high-stakes decisions, especially where the cost of an incorrect or overreaching action is high.

Practical implication: define workflow-level SLOs and review thresholds instead of relying on isolated model quality scores.

NHI Mgmt Group analysis

Agentic AI governance is becoming an identity problem, not just a model-safety problem. Once an agent can act across tools and systems, its permissions, approvals, and revocation lifecycle matter as much as the model itself. That makes the closest governance analogue NHI control, because the system needs scoped entitlement, runtime checks, and lifecycle accountability. Practitioners should treat agent behaviour as an access issue that belongs in IAM and PAM design.

Prompt injection exposes a control gap at the boundary between language and authorization. The article correctly points to jailbreaks and adversarial inputs, but the deeper failure mode is that organisations often let a model's intent become de facto authority. That assumption breaks when the agent can chain tool calls or retrieve sensitive data. The named concept here is authorization drift: the gap between intended permissions and what an agent can actually execute. Practitioners should design for denied-by-default tool access.

Composite monitoring is now a governance requirement for agentic systems. Single-score evaluation does not capture whether a workflow is safe, faithful, and operationally bounded. The article's call for LLM observability is directionally right, but the field needs workflow telemetry that can evidence who accessed what, which tool was used, and when a human approval was bypassed. Practitioners should make agent auditability a control objective, not a reporting afterthought.

Human-in-the-loop cannot be the only safeguard once agents scale beyond low-risk tasks. Human review helps, but it does not solve continuous execution or background delegation. When agents operate across long-running workflows, the review window can close before the risky action occurs. That means policy enforcement must move earlier in the chain, with task-scoped approvals and explicit stop conditions. Practitioners should pair review with enforceable runtime limits.

The market is converging on governance primitives that look more like identity infrastructure than classical AI tooling. Access scope, decision boundaries, audit trails, and revocation are becoming the differentiators that matter in operational environments. That trend will push AI teams to work more closely with IAM, PAM, and security architecture teams. Practitioners should expect agent governance to be bought, designed, and audited like identity control.

What this signals

Agentic AI will force security programmes to merge AI governance with identity governance, because the control questions are now about scope, delegation, and revocation as much as they are about model quality. The most useful near-term pattern is authorization drift: the tendency for agent permissions to expand faster than the task that justified them. That is where runtime policy, auditability, and ownership need to be explicit.

For practitioners, the priority is to make agent behaviour measurable in the same way other privileged systems are measurable. If an agent cannot be tied to a named owner, a bounded permission set, and a complete action trail, it is operating outside governance. The NIST AI Risk Management Framework is relevant here because it pushes teams toward accountable, lifecycle-based control rather than ad hoc guardrails.

The signal for the next 12 months is that organisations will deploy more agents before they mature their control model. That means IAM, PAM, and security architecture teams should prepare for agent identity patterns to appear inside existing workflows, not in separate AI sandboxes. The teams that define policy now will avoid retrofitting controls after the first serious workflow abuse.

For practitioners

• Define task-scoped agent identities Assign each agent workflow the minimum permissions required for a single business function, with explicit boundaries for read, write, and tool invocation. Map those permissions to owned service accounts or workload identities rather than shared credentials.
• Enforce runtime policy at the tool layer Block or approve every external action through policy checks before the agent can call APIs, write records, or retrieve sensitive data. Keep approvals separate from prompt review so language content does not become access authority.
• Build adversarial tests into release gates Simulate prompt injection, jailbreaks, retrieval poisoning, and unsafe tool requests before deployment. Use those tests to prove that the workflow fails closed when instructions conflict with policy.
• Track agent-level audit evidence Log prompts, retrieved sources, tool calls, human approvals, and final actions in a way that supports investigation and compliance review. Ensure the records make it possible to reconstruct why the agent acted, not just what it returned.

Key takeaways

• Agentic workflows change the security problem from model accuracy to runtime authority, because the system can now take actions as well as generate text.
• A large share of current deployments already show rogue behaviour, which means governance gaps are present before many organisations have even formalised policy.
• The control model needs to shift toward scoped agent identity, enforceable tool permissions, and workflow-level auditability before deployment expands further.

Key terms

• Agentic workflow: An agentic workflow is a sequence of tasks executed by an AI agent with some level of tool access and decision authority. In security terms, the workflow matters because it can span multiple systems, identities, and permissions, which makes attribution and revocation harder than with ordinary automation.
• Runtime Authorisation: Runtime authorisation is the practice of deciding access while a task is in progress, rather than only at provisioning time. It matters for NHIs because credentials and entitlements can change risk mid-session, especially when automation or AI agents interact with sensitive systems.
• Permission Drift: Permission drift is the gradual expansion of access beyond what was originally intended. It happens when roles, tokens, and service accounts accumulate unused rights over time, making cloud identities harder to review and more dangerous to compromise.
• Composite Metric: A composite metric combines multiple measures into one view of system performance. For agentic AI, it should blend safety, faithfulness, task success, and business outcome signals so teams can judge whether the workflow is both effective and controlled.

What's in the full article

Fiddler's full blog covers the operational detail this post intentionally leaves for the source:

• Concrete examples of agentic workflows in media, finance, and research operations
• Practical monitoring ideas for hallucination, toxicity, and external attack detection
• A composite metric approach that blends LLM signals with business outcome measures
• Implementation considerations for LLMOps, human review, and scale planning

👉 Fiddler's full blog covers the workflow examples, monitoring approach, and LLMOps planning details.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, machine identity security, and secrets management. It helps security and identity practitioners align runtime access control with real-world workflow risk.