Because models are trained and rewarded mainly for output quality, not for security outcomes. They learn common coding patterns from historical repositories, including insecure ones, and they are rarely optimised against dataflow-heavy weaknesses. Better reasoning helps, but it does not remove the need for independent security controls.
Why model capability and secure code quality can move at different speeds
Model capability improves the model’s ability to predict plausible code, not its ability to optimise for security properties by default. Secure coding is a different target from fluent code completion: it depends on whether the model has been trained, evaluated, and reinforced against insecure patterns, unsafe data handling, weak authorization, and subtle control-flow bugs.
That gap matters because the model can become better at producing syntactically correct or task-complete code while still reproducing the same insecure shortcuts that appear in historical source corpora. When the training signal rewards “looks correct” more than “is secure,” pass rates on secure-code benchmarks tend to plateau even as general reasoning improves.
Improvement is also uneven across vulnerability types. Models often get better at local fixes, obvious input validation, or obvious API misuse, but weaker results persist where security depends on authorization and API security, dataflow, object ownership, state transitions, or multi-step reasoning across components. Those cases are harder to infer from surface patterns alone.
Why training data and evaluation shape the plateau
Most code models inherit a blended lesson from public repositories: useful idioms, repetitive boilerplate, and insecure examples that were never removed from the distribution. If the model is not consistently penalized for insecure output, it can learn to imitate the average repository style rather than the secure one. That is why benchmark gains in general code generation do not automatically translate into secure-code gains.
Another limiting factor is evaluation design. If the benchmark mostly rewards patch correctness or compilation success, the model can improve on the measured target without materially improving security. Security-aware training needs security-aware evaluation, otherwise the model is optimised toward the wrong definition of “good code.”
This is why practitioner teams should treat secure coding benchmarks as a separate control surface, not a side effect of general model progress. For broader governance of secure software delivery, a maturity view such as OWASP SAMM helps teams separate model output quality from secure-development practices.
What actually changes the pass rate
Secure-code pass rates rise when models are given explicit security objectives, robust adversarial evaluation, and downstream checks that catch what generation misses. Independent controls still matter because even a stronger model will not reliably detect every unsafe dependency, privilege assumption, or misuse of secrets in context. If the code touches authentication, access control, or secret handling, independent review remains essential.
In practice, the best results come from layering model-assisted generation with policy, testing, and review. Security-aware prompting can reduce obvious mistakes, but the harder defects are usually found only when the code is validated by tests, static analysis, review gates, and runtime controls that were designed for security rather than style.
That layered approach aligns with NIST SP 800-53 Rev 5 Security and Privacy Controls, especially control families around access control, authentication, audit, configuration, and system integrity. For teams building or governing AI systems, NIST AI Risk Management Framework is also useful for turning “better model” into measurable governance and testing outcomes.
Risk and Threat Considerations
Flat secure-code pass rates create a false sense of progress. Teams may assume that a more capable model is also a safer coding assistant, when in reality the model can still scale insecure patterns faster, produce more convincing vulnerabilities, and widen the blast radius of human review gaps.
Failure mechanism: The model learns from insecure historical code, then generates code that is plausible enough to pass casual review but still embeds weak authorization, unsafe data handling, or fragile control flow. If the organisation measures only general code quality, the security deficit stays hidden.
Impact: Vulnerabilities become cheaper to produce at scale, security teams spend more effort on downstream remediation, and insecure patterns can be replicated across many services before anyone notices the trend.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP ASVS, NIST SP 800-53 Rev 5 and OWASP SAMM set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP ASVS | V8 — Authorization | Secure-code failures often persist in access-control logic and privilege checks. |
| V15 — Secure Coding and Architecture | The question is about why model gains do not convert into secure-code outcomes. | |
| Recommendation — Verify authorization paths and negative cases in generated code before release. Test generated code against secure-design and secure-architecture requirements, not syntax alone. | ||
| NIST SP 800-53 Rev 5 | SA-11 — Developer Testing and Evaluation | Secure-code pass rates depend on security-focused testing beyond general functionality. |
| SI-10 — Information Input Validation | Weak input handling is a common class of defects that models may keep reproducing. | |
| Recommendation — Add security-specific validation to developer testing and acceptance criteria. Validate and sanitize all untrusted inputs in generated code paths. | ||
| OWASP SAMM | Education and Guidance | The plateau reflects a gap between model capability and secure development practice. |
| Recommendation — Embed security guidance into the delivery process so generation and review share the same standards. | ||
Practitioner Guidance
What to prioritise: Treat secure-code quality as its own acceptance criterion. If your evaluation set does not cover dataflow-heavy weaknesses, authentication edge cases, and privilege-sensitive code paths, the pass rate will overstate real security readiness.
What to verify: Check whether the model’s gains hold on security-specific benchmarks, not just general coding tasks. Review whether failures cluster around the same categories, because a flat curve in one vulnerability class often means the training signal is still too weak there.
Common mistake: Assuming that improved reasoning eliminates the need for review. Better reasoning helps, but it does not replace secure design, test coverage, static analysis, or human judgement for high-impact code paths.
Practitioner takeaway: The right question is not whether the model is “better at code,” but whether the surrounding system is strong enough to make insecure code harder to generate, easier to detect, and expensive to ship.
Related resources from NHI Mgmt Group
- How should teams set quality gates when AI-generated code improves overall pass rates but shifts defects into the critical tier?
- Why do AI agents, MCP servers, and model files create compliance risk even when code scans pass?
- Why do APIs create identity risk even when the application code is secure?
- Why do AI agents create new AppSec risk even when code quality improves?