TL;DR: AI code generation can produce working code 83% to 95% of the time, but only 24% to 36% of outputs are both functional and secure, according to Checkmarx-initiated research. Adding a post-generation security review lifts the secure-and-functional rate to 47% to 56%, showing that application security now has to verify what models produce, not just whether it runs.
At a glance
What this is: Checkmarx says AI-generated code can be functional while still failing security expectations, exposing a trust gap that test results alone do not close.
Why it matters: IAM, NHI, and AppSec teams should treat model-produced code as untrusted until independently verified, especially as AI agents and coding workflows expand the attack surface.
By the numbers:
- Frontier models produced working code 83% to 95% of the time on real-world repository tasks.
- Only 24% to 36% of AI-generated solutions were both functional and secure.
- Adding a post-generation security review raised the secure-and-functional rate to 47% to 56%.
Context
AI-generated code changes the AppSec problem from finding defects after software is written to verifying whether the code produced by a model is trustworthy at all. Traditional signals such as compilation success or a passing functional test do not prove that the code is secure, especially when the same class of model is also being asked to review its own output.
That creates a governance gap for application security, because the control point moves earlier in the lifecycle and becomes more dependent on independent verification, context, and accountability. The same dynamic matters for identity-heavy software and AI tooling, where application logic, permissions, and tool access can be introduced faster than human review can keep up.
Key questions
Q: How should teams govern AI-generated code when they cannot review every change?
A: Teams should shift from source-only assurance to runtime assurance. That means correlating deployments, traces, logs, and outcome metrics so behaviour can be validated after code is generated and released. The practical goal is not perfect review coverage, but a dependable record of what the system actually did in production.
Q: Why do AI coding agents require independent verification even when they appear to work well?
A: AI coding agents are probabilistic, so the same prompt can produce different outputs over time. That makes self-review unreliable for quality and security assurance. Independent verification gives teams a repeatable, auditable standard that checks the actual code, not the model’s confidence. This matters most when agents generate large changes or work across many files in one session.
Q: What signs show that AI-driven development is overwhelming AppSec controls?
A: Common signs include review backlogs that keep growing, findings that cannot be prioritised by business impact, and a gap between code creation and human verification. If the organisation can no longer answer which agent or developer changed what, assurance is already lagging behind delivery.
Q: How do organizations balance AI development speed with application security governance?
A: Organizations should make security part of the development loop, not a separate phase. That means setting explicit security requirements in prompts, scanning running applications during development, and verifying that remediations actually hold. The goal is to preserve AI-driven velocity while keeping approval, testing, and accountability aligned with production risk.
Technical breakdown
Why passing tests do not prove secure code
A passing test only proves that code met the specific assertion being checked. It does not prove the code avoided insecure defaults, unsafe data handling, or hidden logic flaws. In AI-generated code, this matters because models often optimise for plausible completion rather than security intent. A repository task may compile cleanly and still include access control mistakes, unsafe dependencies, or assumptions that break under adversarial input. Security therefore cannot be inferred from functional correctness alone. It needs independent verification that examines what the code does not say, not just what it does. Practical implication: treat test success as one signal, not evidence of security.
Practical implication: add independent security review before treating generated code as acceptable.
Independent verification versus model self-review
When the same model class generates and then reviews code, the process loses independence. That is not verification in the security sense, because the reviewer may inherit the same blind spots, confidence patterns, and context gaps as the generator. Effective AppSec requires separate evidence streams, such as deterministic analysis, policy checks, and context-aware review that can challenge the model’s output. The article’s point is not that AI cannot assist, but that AI output needs a control layer that is structurally distinct from the generation layer. Practical implication: separate code generation from security validation so the reviewer is not grading its own work.
Practical implication: keep generation and validation in different control paths.
Why context is now part of the control plane
As AI enters software creation, the attack surface expands across code, dependencies, APIs, permissions, and automated actions. That means security findings cannot be judged only by severity or syntax. Teams need context about reachability, exploitability, ownership, and where the code sits in the broader application and identity model. Without that context, AppSec tools produce either too many findings or too little trust. The control problem is no longer just detection; it is decision quality at speed. Practical implication: connect code, dependency, and ownership context before making remediation decisions.
Practical implication: enrich findings with ownership and exploitability context before triage.
Threat narrative
Attacker objective: The attacker’s objective is to get insecure code accepted as normal software so that exploitable logic, permissions, or dependencies ship into production.
- Entry occurs when AI-generated or AI-modified code introduces insecure logic, unsafe dependencies, or hidden access paths into the software supply chain.
- Credential or control abuse follows when flawed application logic, exposed APIs, or overbroad permissions are carried into the codebase before human review catches them.
- Impact is reached when insecure code is trusted because it functions correctly, allowing vulnerabilities to enter production under the cover of passing tests.
Breaches seen in the wild
- 12,000 secrets in LLM training data: Truffle Security found 11,908 live API keys and passwords hard-coded in web pages captured by Common Crawl, a dataset used to train LLMs.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
AI-generated code creates a trust problem before it creates a vulnerability problem: AppSec teams are being asked to trust outputs that can be syntactically correct, functionally valid, and still insecure. That shifts security from post-build defect detection toward independent validation of model-produced software. The practical conclusion is that trust signals based only on function are no longer sufficient.
The security review must be structurally independent of the generator: if the same model family can both write and judge code, the control is weakened by shared blind spots. Deterministic analysis, policy enforcement, and context-aware triage are stronger because they are not asking the code generator to prove itself. Practitioners should design review paths that can challenge AI output, not merely echo it.
Agentic development widens the AppSec scope beyond code itself: once models can touch files, packages, APIs, and permissions, the control surface includes tool access and workflow behaviour as well as source code. That creates a broader governance problem for IAM, NHI, and AppSec teams working together. The right question is no longer whether AI can help produce code, but whether the organisation can govern the full path from model action to deployable change.
Context quality is becoming a security differentiator: the organisations that can link code, dependencies, ownership, exposure, and remediation history will make better decisions than those that only chase severity. That is the difference between noisy automation and defensible AppSec. Practitioners should treat context enrichment as a control, not a convenience.
What this signals
AI-generated code changes the unit of control: teams can no longer assume that a successful build or test run means the software is trustworthy. The control point has moved to independent validation, where ownership, context, and security policy decide whether model output can enter the software estate.
Context will separate mature programmes from noisy automation: when code, dependencies, permissions, and remediation history are linked, AppSec can prioritise attackable risk instead of drowning in findings. That is especially important as AI-assisted development accelerates the rate at which insecure patterns can be introduced.
AppSec and identity teams now share more of the same problem space: AI coding workflows can create or modify files, packages, APIs, and permissions faster than conventional review loops can follow. Governance has to cover not only source code, but also the tools and identities allowed to change it.
For practitioners
- Implement independent validation for generated code Use a review path that is separate from the model that produced the code, and make deterministic analysis part of the approval gate for AI-assisted changes.
- Treat functional test success as insufficient evidence Require security checks that look for insecure logic, unsafe dependencies, and access control flaws even when the code compiles and passes unit tests.
- Map model access to software-change boundaries Inventory which AI tools, agents, and coding workflows can modify files, install packages, call APIs, or trigger commits, then constrain those actions to the minimum required scope.
- Enrich findings with application context before triage Link findings to ownership, reachability, exploitability, and business criticality so that AppSec teams can prioritise what is actually attackable.
- Prevent AI-assisted change from bypassing governance Make policy checks, review evidence, and remediation history visible in the same workflow that accepts code so teams can trace why a change was approved.
Key takeaways
- AI-generated code can be functionally correct while still failing security expectations, so AppSec cannot rely on build or test success as proof of trustworthiness.
- Independent verification is the control that matters when the generator and the reviewer could otherwise share the same blind spots.
- As AI-assisted development expands, governance must cover code, context, and the identities and tools allowed to make changes.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and MITRE ATT&CK address the attack and risk surface, while NIST CSF 2.0, OWASP ASVS and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | AI coding workflows can misuse identities, tools, and privileges during code creation. |
| Recommendation — Apply ASI03 controls to separate model actions from privileged software-change permissions. | ||
| NIST CSF 2.0 | PR.AA-05 — Access Permissions, Entitlements and Authorizations | AI-assisted development needs governed permissions for code changes and release actions. |
| Recommendation — Enforce PR.AA-05 so AI tools can only change code within explicitly authorised boundaries. | ||
| OWASP ASVS | V15 — Secure Coding and Architecture | The article is about verifying code quality and security beyond functional correctness. |
| Recommendation — Use V15 to verify that code generation pipelines still produce secure architecture decisions. | ||
| CIS Controls v8 | CIS-16 — Application Software Security | The topic centers on application security review and secure software delivery. |
| Recommendation — Apply CIS-16 to embed security checks into the application delivery lifecycle. | ||
| MITRE ATT&CK | TA0006;TA0008 — Credential Access; Lateral Movement | Insecure code can open paths for credential abuse and movement through applications. |
| Recommendation — Map insecure-code findings to TA0006 and TA0008 to prioritise paths that expand attacker reach. | ||
Key terms
- Independent Verification: A trust pattern in which an assertion is checked against a separate source before it is accepted. In identity and security programmes, this prevents fluent but unproven information from being treated as authoritative. It applies to AI outputs, certificates, and other machine-generated claims.
- Attackable Risk: A finding that is not only present but also reachable, exploitable, and relevant in the current environment. Security teams use this concept to separate theoretical defects from issues that can realistically be used to compromise systems or data.
- Context-aware triage: Context-aware triage is the practice of ranking exposed secrets by the access they grant, the systems they touch, and whether they are still active. It is more effective than volume-based scoring because a single privileged credential can create more damage than many low-risk leaks.
- AI-assisted development: A development workflow where an AI system helps generate, refactor, test, or execute code inside the software delivery process. The security issue is not the assistance itself, but that the tool may operate in trusted contexts that also contain credentials, infrastructure access, and other sensitive assets.
Deepen your knowledge
NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. Explore the NHI Foundation Level course, the industry's only accredited NHI security programme, to connect governance patterns with the broader identity and security programme you run.
Published by the NHIMG editorial team on August 19, 2026.
Updated on October 12, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org