What breaks when policy controls are only applied after code is generated?

Why This Matters for Security Teams

Policy applied after code generation creates a false sense of control. By the time a check runs, an AI agent may already have introduced unsafe logic, fetched unapproved packages, exposed secrets, or altered a build pipeline. That shifts security from prevention to remediation, which is a poor fit for agentic workflows that can act quickly and repeatedly. The issue is not just code quality, but governance over what the agent was allowed to do before the output existed. The NIST Cybersecurity Framework 2.0 is useful here because it emphasises governance, risk management, and control placement across the full lifecycle, not only at the end of it.

Security teams often underestimate how much damage can occur before a post-generation policy engine notices a violation. An agent can chain together small permissible steps into a materially unsafe outcome, especially in CI/CD, infrastructure-as-code, or dependency automation. Current guidance suggests that policy must be positioned where decisions are made, not where artefacts are reviewed. In practice, many security teams encounter policy failure only after the generated code has already been merged, built, or deployed, rather than through intentional pre-execution control design.

How It Works in Practice

Effective control design starts with pre-execution checkpoints that evaluate intent, context, and privilege before the agent can generate or modify code. That usually means policy is enforced at prompt intake, tool invocation, dependency selection, file write, and commit stages, rather than only during later scanning. The goal is to stop unsafe actions before they become committed state. This aligns with the broader direction of NIST and OWASP guidance on security-by-design, where control placement matters as much as control content. For AI-specific deployment risk, NIST Cybersecurity Framework 2.0 helps anchor governance, while OWASP guidance for LLM applications highlights prompt and tool abuse patterns that must be constrained before execution.

• Validate the request against approved purpose, repository scope, and allowed tool set before the agent acts.
• Require explicit policy approval for dependency additions, privilege changes, and outbound network calls.
• Use short-lived credentials and scoped tokens so a generated action cannot exceed the intended blast radius.
• Log the pre-action decision, not only the resulting code diff, so reviewers can reconstruct why an action was allowed.

For development pipelines, the strongest pattern is layered prevention: policy at orchestration time, enforcement at tool time, and detection after the fact as a backup. That is especially important when agents can call build systems, secret stores, package managers, or cloud APIs directly. If the only control is a scanner after code generation, the organisation is relying on a checkpoint that cannot stop unsafe dependency resolution, secret exfiltration, or workflow mutation in real time. These controls tend to break down when agents have broad write access to repositories and CI runners because the system can persist harmful changes before any review step runs.

Common Variations and Edge Cases

Tighter pre-execution policy often increases workflow friction, requiring organisations to balance velocity against assurance. That tradeoff is real, but it is usually cheaper than discovering bad code after it has already influenced production systems. Best practice is evolving around how much autonomy should be granted to different classes of agents, and there is no universal standard for this yet. High-trust internal automation may tolerate narrower controls, while internet-facing or build-producing agents should face stricter gates.

Edge cases often appear when teams rely on post-generation policy because they believe reviews, linting, or SAST will catch everything. Those tools are useful, but they do not prevent an agent from choosing a dangerous path, especially when the risk lies in the sequence of actions rather than the final source file. The same problem appears in environments with ephemeral containers, dynamic infrastructure, or multi-step RAG-driven coding assistants, where the harmful decision may be a dependency fetch, configuration mutation, or token use rather than a visible code defect. For that reason, NIST Cybersecurity Framework 2.0 should be paired with runtime guardrails and change approval boundaries, not treated as a downstream review checklist. A practical rule is simple: if an agent can create state, the policy must constrain the creation step, not merely inspect the result.

Related resources from NHI Mgmt Group

• What breaks when AI-generated internal tools are left running after a hackathon?
• What breaks when network controls are used instead of request-level policy for machine access?
• What breaks when IAM controls are applied to autonomous agents without runtime governance?
• How can organisations keep generated identity code aligned with policy?