What breaks when security teams rely only on build-time reviews for AI agent security?

Why This Matters for Security Teams

Build-time review creates a false sense of control when the actual risk appears only after deployment. AI agents are not static applications: they interpret live prompts, consult memory, chain tools, and alter their own next actions based on context. That means a workflow that looked safe in testing can become dangerous once an agent encounters poisoned input, a malformed tool response, or a sensitive dataset it was never intended to touch.

The operational problem is that build-time controls mostly validate code, configuration, and known paths. They do not reliably expose runtime behaviours such as prompt injection, cross-tool escalation, or data leakage triggered by an unexpected sequence of actions. Current guidance from OWASP Agentic AI Top 10 and the NIST AI Risk Management Framework both point to runtime governance as a necessity, not an optional hardening step.

NHIMG research on AI Agents: The New Attack Surface report shows why this matters now: 80% of organisations report their AI agents have already performed actions beyond intended scope, and only 44% have implemented policies to govern them. In practice, many security teams discover agent misuse only after the agent has already accessed data, invoked a tool, or triggered an incident rather than through intentional predeployment testing.

How It Works in Practice

Build-time reviews still matter, but they should be treated as the first gate, not the control plane. For autonomous workloads, the real control point is runtime. Security teams need to evaluate what the agent is trying to do at the moment of action, then decide whether that action is allowed in the current context. That is why intent-based authorisation, short-lived credentials, and workload identity matter more for agents than for traditional apps.

A practical model usually includes four layers. First, issue AI Risk Management Framework-aligned policies that define allowed tasks, data classes, and tool boundaries. Second, bind the agent to a workload identity rather than a human-style account, so the system can prove what the agent is and what execution environment it came from. Third, use just-in-time secrets with short TTLs so credentials expire after the task, not months later. Fourth, enforce policy-as-code at request time so each tool call, retrieval, or data export is checked against live context.

• Use runtime policy engines to validate each action before execution.
• Scope tokens to one task, one session, or one workflow step.
• Separate build-time approval from runtime enforcement.
• Log tool use, memory access, and data egress for investigation and rollback.

Implementation guidance in CSA MAESTRO agentic AI threat modeling framework and OWASP NHI Top 10 reinforces this approach: agent security is not a one-time review, it is continuous control of runtime authority. These controls tend to break down in highly interconnected environments where agents can chain multiple tools across SaaS platforms because the blast radius grows faster than the original review model can track.

Common Variations and Edge Cases

Tighter runtime control often increases latency, policy complexity, and operator overhead, requiring organisations to balance safety against workflow friction. That tradeoff becomes sharper in fast-moving environments such as developer copilots, customer support agents, or multi-agent orchestration pipelines where every additional check can affect user experience.

Best practice is evolving for environments that mix human approval with autonomous execution. Some teams use pre-authorised task templates, while others rely on step-up approval only when the agent crosses a sensitive boundary such as payment actions, credential handling, or production data access. There is no universal standard for this yet, which is why many programmes combine static review with runtime monitoring rather than trying to replace one with the other.

Edge cases also matter. Offline evaluation may be sufficient for low-risk content generation, but it is not enough for agents that can act on behalf of users, alter records, or move laterally between systems. The same applies to memory-enabled assistants, where the risk is not only the initial prompt but also what the agent recalls later and reuses in a different context. NHIMG coverage of Gemini AI Breach - Google Calendar Prompt Injection and CoPhish OAuth Token Theft via Copilot Studio shows how runtime interactions, not code review alone, create the exposure path. The build-time model breaks down when an agent’s permissions, inputs, and tool use change after deployment because the original review no longer matches the live decision surface.

Related resources from NHI Mgmt Group

• What breaks when security teams rely only on DSPM for AI agent governance?
• What breaks when security teams rely on alerts instead of real-time enforcement for AI data protection?
• What breaks when teams rely on a single generic filter for AI agent security?
• What breaks when security teams rely only on human-centric threat models for AI agent activity?