Topic starter
01/08/2026 8:54 pm
TL;DR: Frontier models produced working code 83% to 95% of the time, but secure and functional output reached only 24% to 36%, rising to 47% to 56% with added security steps, according to Checkmarx research. The gap shows why AppSec cannot rely on generation quality alone; deterministic verification and AI-assisted remediation have to work together.
Editorial analysis by NHI Mgmt Group, based on content published by Checkmarx: “Risk, Return, and Remediation: An Investment Strategy for Application Security”.
By the numbers:
- Frontier models produced working code 83% to 95% of the time, but secure and functional output still reached only 24% to 36%.
- Adding security steps raised secure and functional output from 24% to 36% to 47% to 56%.
Key questions
Q: How should AppSec teams govern AI-generated code without relying on the model to validate itself?
A: Use independent deterministic controls for verification and keep them separate from the system that produced the code.
Q: Why does functional code quality not guarantee secure code in AI-assisted development?
A: Because code can run correctly while still containing insecure logic, weak authorization, unsafe assumptions, or unreviewed dependencies.
Q: What are the signs that autonomous coding is outpacing security review capacity?
A: Common signs include a sharp rise in AI-generated pull requests, longer review queues, rushed or shallow reviews, and a growing dependence on review bots to compensate for human overload.
Practitioner guidance
- Separate generation from verification Keep deterministic security checks independent from the model that wrote the code so review remains auditable and repeatable.
- Shift security earlier in the SDLC Move vulnerability detection into the IDE and build stages so fixes happen before flaws propagate into runtime and backlog.
- Measure remediation, not just detection Track how quickly findings become merge-ready fixes, because AI-driven code volume only helps if the organisation can close issues at the same pace.
Bottom line: AI-generated code can be functional without being secure, so generation quality alone is not a valid security control.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
01/08/2026 9:02 pm
Model quality is not a control. High functional output from frontier models does not close the AppSec gap because secure code requires enforcement, not prediction. The article’s core finding is that code generation and code assurance are different disciplines, and organisations that blur them will underinvest in verification. The practitioner conclusion is simple: security gates must remain independent of the model that produced the change.
A few things that frame the scale:
- The average time to mitigate a leaked secret is 36 hours, highlighting the operational burden of manual remediation processes, according to the 2024 State of Secrets Management Survey.
- The average estimated time to remediate a leaked secret is 27 days, despite 75% of organisations expressing strong confidence in their secrets management capabilities, according to the State of Secrets in AppSec.
A question worth separating out:
Q: Should organisations compare AI code generation speed with AppSec remediation speed?
A: Yes, because the real risk is not generation alone but the mismatch between how quickly code is produced and how quickly vulnerabilities are resolved. If remediation cannot keep pace, AI simply increases the volume of insecure changes entering the pipeline. Teams should measure both throughput and fix velocity as linked governance signals.
👉 Read our full editorial: AI-generated code needs hybrid AppSec controls, not model-only security