Why do safety controls and security controls both matter for AI agents?

Why safety and security are both required for AI agents

AI agents sit in a double trust envelope: they must remain aligned with intent, and they must also resist abuse. Safety answers whether the agent’s choices stay acceptable; security answers whether an attacker can steer those choices, steal access, or expand the blast radius. If either side is weak, the system can fail in ways that look very different but are equally consequential.

That is why an agent that is “safe in the lab” can still be dangerous in production if it is exposed to prompt injection, tool abuse, or excessive permissions. It is also why a tightly locked-down agent can still produce harmful outcomes if its goals, memory, or escalation paths are poorly constrained. For AI agents, trustworthy behaviour and defensible control are separate requirements, not substitutes.

What safety controls protect, and where they stop

Safety controls focus on the agent’s behaviour: whether it follows policy, stays within acceptable output boundaries, and avoids actions that are misaligned, deceptive, or operationally reckless. In practice, that means guardrails around planning, instruction hierarchy, memory use, human approval, and the decision to act at all. The goal is not just “good answers”, but behaviour that remains dependable under real workload pressure.

Safety controls work best when the main concern is bad judgement rather than hostile interference. They help when the agent might overstep, mis-rank priorities, fabricate a rationale, or chain too many actions together without sufficient confirmation. But safety controls alone do not stop an attacker from taking advantage of the agent’s authority, especially if the agent can reach tools, data, or downstream systems.

Current guidance in agentic ai security treats safety as one layer in a broader control stack. A practical way to think about it is that safety constrains what the agent should do, while other controls constrain what it can do and under what conditions. The first is about trustworthy behaviour; the second is about defensible authority and exposure. Agentic AI Security Guide frames that layered view clearly.

What security controls protect, and why they are not enough by themselves

Security controls focus on attack resistance: authentication, authorization, least privilege, segmentation, logging, abuse detection, and containment. They reduce the odds that the agent can be hijacked, misused, or turned into a bridge into other systems. For AI agents, this matters because an agent is often an active principal, not just a passive application.

That distinction changes the control problem. If an agent can call APIs, move money, update records, or trigger actions in other services, then a compromise is not just a model issue, it is an operational access issue. An attacker may not need to break the model itself; they may only need to manipulate inputs, reuse a token, or exploit broad permissions. AI Agent Authorisation Guide is useful here because it ties per-action decisions to least privilege and human approval where needed.

Security controls also do not guarantee good outcomes if the agent is allowed to act on bad objectives or stale context. An agent can be hardened against external attack and still make unsafe decisions if the workflow encourages blind automation, uncontrolled memory growth, or unreviewed escalation. In other words, security can preserve the system, but it cannot automatically make the system sensible.

Why the two control sets have to be designed together

The real failure mode is imbalance. Overweight safety and you may get a compliant agent that is still easy to exploit, because the trust boundary is weak. Overweight security and you may get a resilient agent that executes safely from a cyber perspective but still causes business harm through poor judgment, bad automation choices, or incorrect delegation. The control objective is to keep both the decision quality and the attack surface within acceptable bounds.

That is why agent design should treat policy, authority, and observability as connected decisions. If the agent can take action, the action should be bounded. If the agent can remember, the memory should be governed. If the agent can reach tools, the tools should be permissioned per use case. And if the agent can affect business processes, its behaviour should be measurable enough to detect drift, misuse, or silent failure. Zero Trust for AI Agents is a strong reference point for that design pattern.

For teams building or approving agents, the question is not which layer matters more. The better question is whether the agent’s authority is narrow enough that a mistake is survivable, and whether its behaviour is constrained enough that a compromise is detectable before it becomes a business incident.

Risk and Threat Considerations

AI agents are especially exposed because the same capability that makes them useful, tool access plus delegated action, also makes them attractive to attackers. Prompt injection, token theft, excessive permissions, and unsafe delegation can turn an otherwise helpful agent into an execution path for data exfiltration, fraud, or destructive change. A system can look aligned and still be one malformed prompt away from misuse.

Failure mechanism: An attacker or malicious input steers the agent’s decisions, reuses its credentials, or abuses its tool access so that the agent performs actions outside the user’s or organisation’s intent.

Impact: The result can be unauthorized data access, corrupted business actions, lateral movement into connected systems, or loss of trust in the agent’s decisions even when the underlying model remains technically available.

Practitioner Guidance

What to prioritise: Treat the agent’s action boundary as the main control surface. If the agent can do more than answer, every permitted action should be explicitly bounded by scope, approval path, and monitoring, not assumed safe because the model is “well behaved”.

What to verify: Confirm that safety review and security review are both represented before launch. A good test is whether you can explain, for each high-impact action, what prevents unsafe reasoning, what prevents unauthorized execution, and what evidence would show either one failing.

Common mistake: Teams often try to solve agent risk with only content guardrails or only IAM-style restrictions. That split fails because AI agents need both behavioural constraints and access constraints to stay trustworthy under real-world pressure.

Practitioner takeaway: The durable design principle is to make unsafe behaviour hard and unauthorized action harder, because either failure mode can sink an AI agent even when the other looks sound.

Related resources from NHI Mgmt Group

• What are the security risks associated with AI agents?
• How should security teams manage permissions for AI agents?
• How should security teams govern AI agents that use OAuth access?
• How should security teams limit the risk from AI agents that have access to production systems?