How do you reduce the chance of an AI agent taking unsafe actions?

Use layered controls rather than a single approval step. Filter hostile input before the model runs, validate outputs before any side effect, and reserve human approval for high-impact actions such as spending, deleting, or policy changes. That combination reduces both prompt-injection risk and accidental tool misuse.

Why This Matters for Security Teams

AI agents reduce friction only when they can act, and that is exactly why they create operational risk. Once an agent can call tools, chain steps, or retry failed actions, a single bad prompt can turn into a real-world side effect. Static role-based access is often too blunt for this environment because it assumes predictable human workflows, not autonomous goal-seeking behaviour. Guidance from the OWASP Agentic AI Top 10 and the NIST AI Risk Management Framework both point toward layered controls, contextual policy, and explicit oversight for high-impact actions.

NHIMG research shows why this matters in practice: in the AI Agents: The New Attack Surface report, 80% of organisations said their AI agents had already acted beyond intended scope, including accessing unauthorised systems, sharing sensitive data, and revealing credentials. That is not a theoretical policy gap; it is an execution problem that appears after deployment. Teams that wait for a human approval prompt on every action often discover too late that the agent has already gathered context, opened tools, or staged the next step. In practice, many security teams encounter unsafe agent behaviour only after the first unintended side effect has already occurred, rather than through intentional testing.

How It Works in Practice

The most reliable pattern is to treat the agent like a high-risk workload with tightly scoped authority, not like a human user with a standing role. Start by limiting what the agent can see, then limit what it can do, then limit when it can do it. That means input filtering for prompt injection, runtime policy checks before each tool call, output validation before side effects, and just-in-time approval only for actions that are expensive, irreversible, or legally sensitive. The OWASP NHI Top 10 is a useful reference point because agent risk is often really identity and privilege risk.

• Use workload identity for the agent itself, so the system proves what it is before any tool access is granted.
• Issue ephemeral credentials per task, not long-lived secrets that remain valid after the current objective ends.
• Evaluate policy at request time with context such as target system, data sensitivity, and action impact.
• Separate read, write, and destructive capabilities so a planning step cannot silently become an execution step.
• Log the full decision path, including tool calls and policy denials, for post-incident review.

This approach aligns with current implementation guidance from the CSA MAESTRO agentic AI threat modeling framework, which emphasizes modelling the agent’s actions, not only its inputs. For infrastructure teams, the key distinction is that short-lived identity and real-time policy are more important than static RBAC alone. These controls tend to break down when agents are allowed to chain tools across multiple tenants or shared service accounts because privilege boundaries become ambiguous and approval signals arrive too late.

Common Variations and Edge Cases

Tighter approval controls often increase latency and operator overhead, so organisations must balance safety against task completion speed. Best practice is evolving, and there is no universal standard for agent approvals yet, especially for low-risk automation versus high-impact workflows. For example, a customer-support agent may be allowed to draft responses automatically, while a finance agent should require explicit approval before payment initiation. The correct control depends on blast radius, not on whether the model is “trusted.”

Edge cases matter most when the agent works across multiple tools or data domains. If an agent can search email, open tickets, query databases, and execute scripts, the security problem becomes compound: each tool may be safe in isolation, but unsafe in sequence. The Analysis of Claude Code Security highlights how code-facing agents can turn normal developer workflows into privileged execution paths. That is why validation must happen at the boundary of every side effect, not just at the model prompt. Current guidance suggests reserving human approval for irreversible actions, while allowing bounded automation for routine steps that can be rolled back. The toughest failures appear in shared-service environments where one agent identity spans many tools, because a single compromise can cascade across systems before the approval layer is even engaged.

Related resources from NHI Mgmt Group

• When do AI agent credentials create more risk than they reduce?
• What is the difference between human identity governance and AI agent governance?
• When does just-in-time access reduce risk for agentic AI, and when does it fall short?
• When does AI agent access create more risk than it reduces?