What breaks when AI-assisted code is merged without provenance controls?

Why This Matters for Security Teams

When AI-assisted code lands in a repository without provenance controls, the immediate problem is not just “who wrote this line.” The deeper issue is whether the organisation can prove what was reviewed, what was generated, and what dependencies or prompts influenced the final artifact. Without that chain of custody, security teams lose defensible evidence for audit, incident response, and policy enforcement. Guidance from NIST SP 800-53 Rev 5 Security and Privacy Controls reinforces that traceability and accountability are not optional once code is promoted into controlled environments.

The risk is amplified by modern secret exposure patterns. NHIMG’s The State of Secrets in AppSec highlights that the average estimated time to remediate a leaked secret is 27 days, which means provenance gaps can overlap with real exposure windows long enough for attackers to exploit them. If AI-assisted code also introduces hidden tokens, unsafe libraries, or copied insecure patterns, teams may only discover the source after the damage is already live.

In practice, many security teams encounter provenance failures only after an incident review has already shown they cannot reconstruct how the bad code entered production.

How It Works in Practice

Provenance controls answer three practical questions: where the code came from, what transformed it, and who approved its inclusion. For AI-assisted development, that usually means capturing the originating model or tool, the prompt or task context where allowed, the human reviewer, and the build pipeline evidence that ties the merged commit back to a verified artifact. This is not the same as “just keep good tickets.” Ticket metadata is helpful, but it does not establish cryptographic or procedural lineage.

Security teams typically combine source control controls with build-system attestations, signed commits, and policy checks that block unverified artifacts. The aim is to preserve enough evidence to support later validation of authorship, review, and dependency integrity. In operational terms, this works best when provenance is collected at generation time, then preserved through CI/CD and release promotion rather than reconstructed after the fact.

NHIMG’s DeepSeek breach is a useful reminder that exposed systems and embedded secrets can coexist with weak control boundaries, which is why provenance and secret detection should be treated as complementary controls, not separate silos.

• Record whether a human, an AI assistant, or both contributed to the change.
• Require signed build outputs or attestations before merge.
• Bind code review evidence to the exact commit and dependency set.
• Preserve prompt or generation metadata where policy permits.
• Block merges when lineage is missing, altered, or unverifiable.

These controls tend to break down when teams allow direct-to-main changes from ad hoc AI plugins because the review and attribution trail never exists in a verifiable form.

Common Variations and Edge Cases

Tighter provenance controls often increase developer friction, requiring organisations to balance traceability against delivery speed. That tradeoff becomes more visible in fast-moving teams, but current guidance suggests the cost of weaker evidence is usually higher during incidents, audits, and third-party reviews.

There is no universal standard for proving AI-assisted authorship yet. Some teams use lightweight metadata tags, while others require full software supply chain attestations. The right level depends on the regulatory burden, the sensitivity of the codebase, and whether the system handles secrets or regulated data. Best practice is evolving, but the direction is clear: provenance should be tamper-evident and reviewable.

Edge cases matter. A developer may paste AI-generated snippets into a local branch, or a plugin may rewrite code as part of a refactor. In those cases, provenance controls should distinguish between assistance and authorship without turning every merge into a forensic exercise. The goal is not to ban AI-assisted coding, but to ensure investigators can still answer what changed, who approved it, and whether the final artifact was altered after review.

For broader NHI and software integrity context, NHIMG’s Ultimate Guide to NHIs — Standards is a useful reference point for identity and control alignment across machine actors and software supply chains.

Related resources from NHI Mgmt Group

• How should security teams use AI in secret scanning without creating new blind spots?
• How should security teams monitor AI agent activity without disrupting developers?
• What breaks when employees use AI tools inside browser sessions without data controls?
• What breaks when AI workloads scale without lifecycle controls?