What breaks when AI-generated changes are reviewed only at merge time?

Why This Matters for Security Teams

Merge-time review sounds efficient, but it assumes the risky part of delivery is the final diff. With AI-generated changes, the real exposure often begins earlier, when a prompt, model output, or agent action creates code, infrastructure, secrets handling, or workflow logic in the same motion. That means security is no longer just validating a pull request. It has to govern how artefacts are produced, whether dependencies are trusted, and what the pipeline is allowed to execute.

This matters because merge approval is a control point, not a control strategy. A reviewer can miss embedded misuse of permissions, unsafe defaults, hidden dependency changes, or an agent that has already propagated a flawed pattern into multiple repositories. The practical question is not whether a human can spot a bad diff. It is whether the organisation can prevent unsafe artefacts from being generated, promoted, and deployed before review ever starts. The NIST Cybersecurity Framework 2.0 is useful here because it pushes teams to think in terms of govern, protect, detect, respond, and recover, rather than treating approval as the whole control plane.

In practice, many security teams encounter the problem only after an AI-generated change has already landed in build systems, configuration drift, or downstream environments rather than through intentional review of the generation step.

How It Works in Practice

When AI-generated changes are reviewed only at merge time, several security checks arrive too late. A model may have produced code that is syntactically valid but structurally unsafe, such as permissive IAM policy updates, insecure dependency pins, or workflow steps that expose credentials to logs. If the review process only sees the final pull request, it misses the provenance of the change and the chain of decisions that created it.

Operationally, stronger controls need to exist at three points: generation, build, and runtime. At generation time, teams should constrain prompts, validate tool access, and log the identity of the actor or agent that initiated the change. At build time, they should inspect artefacts, scan dependencies, and compare outputs against policy. At runtime, they need monitoring for drift, abuse, and unexpected privilege use. This is consistent with the intent of NIST AI 100-1 and related AI risk guidance, which emphasise accountability, traceability, and lifecycle controls rather than one-off approval gates.

• Record prompt, model, and tool provenance for AI-assisted changes.
• Scan generated code, dependencies, and infrastructure definitions before merge.
• Require policy checks on secrets exposure, privilege escalation, and external calls.
• Revalidate deployed artefacts because post-merge drift can invalidate the original approval.

Where AI agents are involved, the review model also needs to account for non-human identity governance: which agent acted, what permissions it had, and whether its execution authority was bounded. Guidance from OWASP guidance for LLM applications is relevant because prompt injection, tool misuse, and output manipulation can all produce changes that appear legitimate at review time. These controls tend to break down in fast-moving monorepos with auto-generated infrastructure and shared deployment credentials because the same pipeline that creates the change can also promote it before any meaningful inspection occurs.

Common Variations and Edge Cases

Tighter review of AI-generated changes often increases delivery overhead, requiring organisations to balance speed against confidence. That tradeoff is real, especially where teams generate large numbers of low-risk changes and do not want to slow every release with heavy manual inspection.

Best practice is evolving on how much can be automated versus manually reviewed. For low-risk code paths, policy-as-code, automated diff analysis, and signed artefact checks may be enough. For changes that affect authentication, secrets, network policy, or production infrastructure, current guidance suggests adding stricter human approval and stronger provenance controls. The distinction matters because an AI-generated edit to a test helper is not equivalent to an AI-generated update to a deployment manifest or access policy.

Edge cases also appear when the model produces several small changes that are individually acceptable but collectively unsafe. Another common failure mode is a “shadow approval” pattern, where a reviewer focuses on style or syntax while missing that the agent has introduced a new dependency, a new outbound integration, or a new privilege path. For regulated environments, the relevant question is not just whether the merge passed. It is whether the organisation can prove what was generated, by whom or by what, and under what controls. In multi-agent or highly automated environments, merge review alone is too narrow because the security decision may have already been made upstream.

Related resources from NHI Mgmt Group

• What breaks when AI-generated code is reviewed without security gates?
• When do AI-generated changes become a workload identity problem?
• Why do AI-generated code changes increase application security risk?
• What breaks when AI agent access is reviewed only after the fact?