GenAI code creation is creating a new AppSec gap

At a glance

What this is: Checkmarx argues that generative AI has moved the software supply chain upstream into code creation, exposing a security gap that traditional post-commit AppSec tools do not cover.

Why it matters: For IAM and NHI practitioners, the same shift reinforces a broader control lesson: governance must attach to the point of creation, not just to review and remediation after the fact.

By the numbers:

• 99% of development teams now use AI-assisted coding tools, while only 29% have implemented formal AI security controls.
• 60% of organisations say GenAI usage is unapproved, yet still occurs.
• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap.

👉 Read Checkmarx's analysis of AI-generated code and the new AppSec gap

Context

Generative AI code assistants are changing the software supply chain by moving risk to the moment code is created. That means the primary control challenge is no longer limited to post-commit scanning or CI gates, because unsafe logic, insecure defaults, and unvetted dependencies can be introduced before a repository ever sees the code. For identity programmes, the underlying lesson is familiar: governance fails when it is attached too late in the lifecycle.

This matters because AI-assisted development is now a human and machine collaboration problem, not just a secure coding problem. When an AI assistant generates or rewrites code, security teams need visibility into the creation step, the approval step, and the policy step, especially where secrets, access paths, or third-party dependencies are involved. The article's starting point is increasingly typical, not exceptional, across development teams.

Checkmarx's analysis frames the IDE as the new security boundary for AI-assisted development, which is consistent with the broader shift toward earlier policy enforcement in modern engineering workflows. Where code generation is effectively a non-human contributor, governance needs to treat the assistant as part of the production chain, not as an invisible productivity layer.

Key questions

Q: How should security teams govern AI-assisted coding tools in the IDE?

A: They should treat AI-assisted coding tools as part of the control surface, not just a developer convenience. That means applying policy at the point of code generation, controlling which plugins and packages can run, and logging every allow, block, and override with identity and context. The goal is to keep untrusted code from becoming trusted software.

Q: Why do AI-generated code pipelines create more security risk than traditional development?

A: They compress the time between defect creation, discovery, and exploitation. When code and exploits can be produced faster than human review cycles, the main risk is not just more vulnerabilities. It is more reachable vulnerabilities surviving long enough to become incidents.

Q: What do security teams get wrong about post-commit AppSec scanning?

A: They often assume post-commit scanning is the primary control, when it is actually a late-stage detection layer. In AI-assisted workflows, many risky choices happen earlier, so a scan can confirm a problem has already entered the pipeline rather than preventing it. Prevention has to move upstream.

Q: How can organisations prove their AI controls are actually working?

A: Look for evidence that policy decisions are logged, sensitive prompts are being redacted or blocked when required, and approved AI interactions are traceable by identity and business context. Effective programmes produce audit-ready records, not just policy text. If the control cannot explain what happened in a session, it is not operational enough.

Technical breakdown

Why post-commit scanning misses AI-generated risk

Traditional AppSec tools assume code is written by humans, committed, and then scanned before release. AI-assisted coding breaks that sequence because vulnerabilities can be introduced during generation, not after review. By the time SAST or SCA runs, the risky pattern may already be merged, deployed, or buried inside a larger refactor. The deeper problem is context loss: scanners see the final code, but not the prompt, the suggestion, or the assistant behaviour that created it. That makes detection later, slower, and less precise.

Practical implication: Shift validation left into the IDE so risky patterns are flagged while code is being authored.

How LLM-generated code changes the software supply chain

In a GenAI workflow, the software supply chain now starts with prompt input rather than with source control. The assistant can introduce scaffolding, dependency suggestions, and refactoring patterns that become part of the build even if no one treats them as supply chain inputs. This expands the attack surface upstream, because a malformed suggestion can influence libraries, logic flow, and security assumptions before any formal review. The result is a new trust boundary around AI-generated contributions, where provenance and policy matter as much as syntax.

Practical implication: Treat AI suggestions as supply chain inputs and apply controls to generation, not just to repository content.

Shadow AI and security drift in developer workflows

When developers use unsanctioned assistants or bypass approved workflows, organisations lose visibility into what tools are shaping production code. That creates security drift, which is the gradual divergence between policy and actual practice. In this state, AI-generated logic, risky dependencies, and even secrets can bypass expected review paths. The issue is not only technical. It is governance failure at the workflow level, because the organisation no longer knows which systems are influencing code quality or security outcomes.

Practical implication: Inventory sanctioned assistants, log usage, and enforce policy at the point of authoring.

Threat narrative

Attacker objective: The objective is to get insecure or malicious code into production by exploiting the trust developers place in AI-generated output.

1. Entry occurs when developers feed prompts into AI coding assistants that generate boilerplate, refactoring, or full functions inside the IDE.
2. Escalation happens when unsafe defaults, hallucinated dependencies, or insecure logic are accepted and committed without equivalent human review.
3. Impact follows when vulnerable code reaches production with hidden provenance, making remediation slower and harder than in traditional post-commit pipelines.

NHI Mgmt Group analysis

AI-assisted coding has created an upstream governance problem, not just a coding-quality problem. Once generation becomes part of the software supply chain, security policy has to move to the point where code is created, not where it is finally reviewed. Post-commit controls still matter, but they now sit too late in the lifecycle to catch the full class of AI-introduced issues. The practitioner conclusion is straightforward: control the creation layer or accept blind spots.

Prompt-time validation is the named control gap this article exposes. The article's core insight is that the IDE is now a security boundary, because prompts, suggestions, and assistant output shape production code before traditional AppSec tooling sees it. That means the old assumption that code provenance can be reconstructed from repository history is weakening. Security teams should treat the generation step as a governable event, not an invisible convenience.

Security drift is what happens when GenAI usage expands faster than policy. Checkmarx's figures show the scale of the mismatch: nearly universal adoption alongside limited formal controls. That combination creates a programme-level gap where teams believe review exists, but authoring has already escaped central oversight. The lesson for broader identity and access governance is that untracked creation pathways become shadow pathways, whether the artefact is code, secrets, or privileges.

The identity intersection is real because AI assistants are becoming non-human contributors inside delivery pipelines. Once assistants shape code, dependency selection, and remediation paths, their actions deserve governance logic closer to NHI and agentic AI controls than to ordinary tooling oversight. That does not mean treating every assistant as fully autonomous, but it does mean binding it to policy, auditability, and approval boundaries. The practitioner conclusion is to govern the contributor, not just the output.

Named concept: prompt-time supply chain exposure. This describes the security risk introduced when AI-generated content becomes part of the development supply chain before a repository scan or human review occurs. It matters because it shifts the attack surface into the authoring workflow itself, where traditional scanners have no visibility into intent. Practitioners should align controls to this earlier boundary rather than assuming downstream inspection is sufficient.

What this signals

AI-assisted development is becoming a governance problem as much as a productivity problem. As generation moves into the IDE, teams need to decide whether they are managing code output or managing a non-human contributor embedded in the delivery chain. The practical signal is that policy, auditability, and approval boundaries now have to reach the creation layer, not just the release layer.

Prompt-time supply chain exposure: this is the operational gap created when AI suggestions enter development before repository controls can inspect them. Organisations that still rely on post-commit detection will continue to see issues surface too late for efficient remediation. The stronger model is to bind security checks to authoring, with visibility into assistant usage and generated dependencies.

For identity and access leaders, the broader implication is that untracked creation pathways behave like shadow control planes. Where tools can generate code, suggest packages, or shape remediation logic without central oversight, the programme has a governance blind spot. Map that gap against NIST Cybersecurity Framework 2.0 and the OWASP Non-Human Identity Top 10 to anchor policy in an external control reference.

For practitioners

• Implement IDE-level security validation Add controls that inspect code as it is typed or generated, with checks for insecure APIs, dangerous defaults, and risky dependency suggestions before commit. This is the only way to catch prompt-time supply chain exposure early enough for meaningful intervention.
• Inventory and sanction AI coding assistants Document which assistants are approved, which teams are using them, and where shadow AI is already present. Tie that inventory to policy enforcement so unapproved tools cannot quietly shape production code.
• Extend secrets and dependency scanning to generated code Apply SCA and secrets detection to AI-produced snippets, not just to repository commits, so hidden packages and exposed credentials are identified in the authoring flow. This helps reduce the blind spot created when code appears valid but was generated outside normal review.
• Measure pre-commit versus post-commit findings Track how many issues are caught before code leaves the IDE versus after repository submission, then use the delta to prove whether upstream controls are actually reducing risk. If the ratio does not improve, the control design is still too late.

Key takeaways

• Generative AI has moved AppSec risk upstream into code creation, where post-commit scanners arrive too late.
• Checkmarx's survey data shows near-universal AI-assisted coding adoption alongside limited formal AI security controls.
• The practical response is to govern the IDE, not just the repository, with policy, visibility, and validation at the point of generation.

Key terms

• Prompt-time supply chain exposure: The risk created when AI-generated code, suggestions, or dependencies enter the development process before traditional repository controls can inspect them. It shifts security concerns from downstream review to the authoring moment, where provenance is weaker and visibility is limited.
• Scope drift: Scope drift is the gradual mismatch between what an integration was meant to do and what its credentials still allow it to do. It happens when permissions are not revalidated as business needs change, creating hidden over-privilege across SaaS and API-connected systems.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.

What's in the full article

Checkmarx's full article covers the operational detail this post intentionally leaves for the source:

• Examples of insecure defaults and logic flaws that AI-generated code can introduce before commit
• The article's breakdown of post-commit scanning latency and why it fails in AI-assisted workflows
• Operational guidance on shifting controls into the IDE and validating intent as code is written
• Checkmarx's discussion of shadow AI and governance drift across developer teams

👉 The full Checkmarx article covers prompt-time risk, IDE controls, and the developer workflow changes behind this security gap.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management. It helps security practitioners connect identity controls to the broader governance problems created by non-human contributors.