How should security teams design AI security controls when agentic systems can escalate beyond their intended task scope?

Why This Matters for Security Teams

Agentic systems are not just better chat interfaces. Once an AI agent can call tools, chain actions, and retain state across steps, it becomes an execution layer that can drift beyond the original task. That changes the control problem from prompt quality to containment, authorization, and monitoring. The most useful starting point is the NIST AI Risk Management Framework, because it frames AI risk as a governance and lifecycle issue rather than a one-time deployment decision.

Security teams often underestimate how quickly an agent can combine harmless permissions into an unsafe outcome. A model that can read a ticket, query a database, open a browser, or invoke an internal API may still be operating within each isolated permission while violating the intended task boundary overall. That is why control design has to treat tool access, identity, data scope, and action approval as one system. This is also where NHI governance becomes relevant, because agents frequently rely on non-human credentials, scoped tokens, and service identities that outlive the original workflow if not tightly managed.

In practice, many security teams encounter agentic overreach only after an internal tool has already been used in an unintended sequence, rather than through intentional design review.

How It Works in Practice

Practical control design starts by separating what the agent may observe from what it may change. Observation can often be broader than execution, but execution should be constrained to the smallest useful set of tools, actions, and identities. Current guidance from frameworks such as the CSA MAESTRO agentic AI threat modeling framework and the OWASP Agentic AI Top 10 points toward layered controls rather than a single “safe prompt” pattern.

That usually means:

• Binding every tool call to an authenticated agent identity with narrowly scoped permissions.
• Segmenting network and data access so the agent cannot discover or pivot into unrelated systems.
• Requiring human approval for destructive, irreversible, or high-value actions.
• Logging prompts, tool calls, retrieved context, and outputs so reviewers can reconstruct decision paths.
• Using policy checks before and after execution to validate intent, data classification, and destination.

The best practice is evolving, but the operational principle is clear: trust decisions should be enforced in code, policy engines, and infrastructure, not just in natural-language instructions. Teams should also model prompt injection, indirect tool abuse, and data exfiltration as control failures, not just “bad model behavior.” The MITRE ATLAS adversarial AI threat matrix is useful here because it helps map how agents can be manipulated at inference time or during orchestration. These controls tend to break down in environments where the agent is given broad API credentials, shared runtime roles, or direct access to production systems without action-level approval.

Common Variations and Edge Cases

Tighter containment often increases operational overhead, requiring organisations to balance autonomy against review latency and workflow friction. That tradeoff is especially visible in customer support, DevOps, and SOC automation, where teams want speed but cannot safely grant open-ended execution. In low-risk use cases, a read-only agent with constrained retrieval may be sufficient. In higher-risk environments, current guidance suggests moving to staged execution, where the agent proposes actions, a policy engine validates them, and a separate identity performs the final commit.

There is no universal standard for this yet, so teams should be explicit about the boundary they are choosing. For example, if an agent can create tickets but not deploy code, that boundary must be enforced by separate credentials, separate network paths, and separate audit trails. The OWASP Non-Human Identity Top 10 is particularly relevant where agents rely on long-lived service identities or secrets. The same applies when multiple agents cooperate, because one compromised agent can inherit authority through shared context or delegated tokens. In practice, the hardest failures appear when teams assume a “helper” agent will stay narrow, but the surrounding identity and infrastructure design silently gives it room to escalate.

Related resources from NHI Mgmt Group

• How should security teams design challenge-response controls against agentic AI automation?
• How should security teams control context in agentic AI systems?
• How should security teams score vulnerabilities in agentic AI systems?
• How should security teams evaluate AI red teaming vendors for agentic systems?