GPT-5 code output raises the cost of verifying AI-generated software

At a glance

What this is: Sonar's update finds that GPT-5 generates stronger benchmark results, but the code it produces is still unusually complex and carries a high verification burden.

Why it matters: This matters because AI-assisted software delivery shifts the control point from authoring to verification, and IAM, PAM, NHI, and security teams should treat AI-generated code as untrusted until it passes the same checks as human code.

By the numbers:

• GPT-5-minimal introduced 3.90 issues for every correct solution, nearly double the rate of Claude Sonnet 4.
• GPT-5-minimal produced only 60 total vulnerabilities, compared with 141 for Claude Sonnet 4.
• GPT-5-minimal had a code smell density of 25.28 per KLOC, higher than every model except OpenCoder-8B.

Context

GPT-5 output quality is not just a model-performance question, it is a software governance question. When an AI system can generate code that passes tests but still carries high complexity and maintainability debt, existing review models become too shallow if they focus only on correctness. For IAM and security teams, the same lesson applies to AI-assisted code that touches authentication, secrets handling, and policy logic, where functional success can still hide unsafe implementation choices.

Sonar's update sits squarely in the gap between benchmark performance and production readiness. The article shows that better pass rates do not automatically produce cleaner code, lower review effort, or lower operational risk. That is a familiar pattern in software assurance: the closer code gets to working, the easier it is to overlook structural defects that later create security, resilience, and maintenance problems.

Key questions

Q: What should teams do first when AI-generated code passes tests but still looks risky?

A: Start with automated static analysis and complexity review before any human approval. Passing tests only show that the code solved the sample task, not that it is maintainable or safe enough for production. The first control should therefore be to scan, score, and gate the output before it enters the merge or release path.

Q: Why does AI-generated code still require strong security governance?

A: Because lower defect rates do not remove business context, deployment risk, or ownership. AI can surface issues faster, but teams still need to decide whether a fix is safe, whether a vulnerability is exploitable in their environment, and whether remediation should interrupt production. Governance remains the control that turns findings into accountable action.

Q: What are the signs that generative AI code is failing a security review?

A: The clearest signs are code that passes functional tests but introduces unsafe input handling, string concatenation with user data, or missing sanitisation. Another warning sign is when developers need significant manual rework after generation, which erodes the productivity benefit. If remediation becomes the real work, the AI output is not meeting the security objective.

Q: How should security teams balance speed and verification in AI-assisted development?

A: Use AI to accelerate drafting, but keep the release decision grounded in the same assurance standards used for human-written code. The practical balance is not faster approval, but faster initial generation followed by consistent policy checks, human review where needed, and automatic rejection of output that exceeds quality thresholds.

Technical breakdown

Why benchmark pass rates do not equal production-ready code

A benchmark pass rate measures whether generated code solves the task, not whether it is maintainable, readable, or safe to operate. LLMs can optimise for short-horizon functional success while still producing convoluted control flow, duplicated logic, and brittle error handling. That is why a model can look strong in tests and still create costly review and refactoring work after the fact. In security terms, the control problem moves from “did it work once?” to “can humans and automated scanners reason about it later?”

Practical implication: Treat benchmark performance as a selection input, not a release gate, and require static analysis before any AI-generated code reaches review or merge.

Code smells, cyclomatic complexity, and review fatigue

Code smells are maintainability indicators, not necessarily security findings. Cyclomatic complexity measures the number of decision paths in code, while cognitive complexity estimates how hard it is for a person to understand the logic. When both rise sharply, reviewers spend more time reconstructing intent and less time validating edge cases, security checks, and failure handling. That creates review fatigue, where teams become more likely to miss subtle defects because the code is simply harder to read. The article’s core point is that verbosity and complexity are governance liabilities, not cosmetic issues.

Practical implication: Set complexity thresholds for AI-generated code and route over-complex output into mandatory refactoring before approval.

Why low vulnerability counts still leave a security gap

A low vulnerability count does not mean the code is safe enough for production. Security defects are only one category of risk, and AI-generated software can still contain path traversal, injection, and control-flow mistakes even when the overall vulnerability rate is comparatively low. In practice, organisations should separate ‘fewer exploitable flaws’ from ‘acceptable software quality’, because maintainability debt often becomes the route by which security bugs survive future changes. This is especially relevant where generated code implements access control or handles secrets, because complexity makes later assurance harder.

Practical implication: Use security scanning plus maintainability review together, and do not waive refactoring just because vulnerability density looks low.

NHI Mgmt Group analysis

AI code generation has a verification problem, not just a correctness problem. The article shows that stronger benchmark output can coexist with poor maintainability and a high issue burden. That means the real control objective is not whether the model can write working code, but whether the organisation can verify it at production standards before the code becomes part of the attack surface. The practitioner conclusion is to govern AI output as untrusted software until it is explicitly validated.

Complexity is the hidden tax that most AI coding programmes underestimate. GPT-5-minimal’s higher code volume and complexity mean the downstream cost is not just review time, but reduced reviewer attention on the exact defects that matter most. Complex generated code also compounds future change risk, because each fix must be interpreted through more branches and more implicit behaviour. The practitioner conclusion is to treat complexity budgets as part of secure SDLC governance.

Security findings can mask broader software quality failure. A lower vulnerability count does not offset the maintainability burden created by code smells, control-flow mistakes, and verbose implementations. That distinction matters in identity-heavy code paths, where authentication logic, token handling, and policy evaluation are often embedded in generated functions. The practitioner conclusion is that security and maintainability must be tested together, not traded off against each other.

AI-generated code requires policy-based verification, not model trust. The article’s strongest signal is that model evolution does not eliminate the need for repeatable controls across all output. Static analysis, review thresholds, and refactoring rules should apply regardless of which LLM produced the code, because the governance challenge is the output profile, not the brand of the model. The practitioner conclusion is to standardise verification across all AI coding tools.

Identity and access logic deserve extra scrutiny in AI-assisted development. Where generated code touches secrets, authentication, or authorization, complexity becomes an operational risk multiplier because defects are harder to detect and harder to fix safely. This intersects directly with IAM and NHI governance when code creates, stores, or checks credentials for humans, services, or agents. The practitioner conclusion is to apply the strictest review path to identity-critical code paths.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Identity Security Buyer's Guide

What this signals

AI code generation is creating an assurance gap, not just a productivity gain. As models improve, the security question shifts from whether they can produce working code to whether organisations can still verify that code at scale. In identity-heavy systems, where generated logic may touch authentication, authorization, and secrets, that gap becomes a programme risk rather than a developer convenience.

AI-generated code should be governed as a new software supply chain tier. The verification problem now sits between prompt and production, which means security teams need policy gates, static analysis, and human review criteria that are independent of the model brand. The more code an LLM emits, the more important it becomes to define where automated trust ends and operational accountability begins.

Model improvement does not remove the need for security and maintainability thresholds. Sonar’s update shows that AI output can still carry a heavy quality cost even when functional results improve, and our research shows AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026. For practitioners, the signal is clear: verification must keep pace with AI adoption.

For practitioners

• Apply static analysis to every AI-generated change Require automated static analysis on all generated code before merge, with no exceptions for passing test cases or familiar models.
• Set complexity thresholds for generated code Reject or refactor output that exceeds agreed cyclomatic complexity and cognitive complexity limits, even when unit tests pass.
• Add human review to identity-critical paths Route authentication, authorization, and secrets-handling code through a stricter review path than ordinary application logic.
• Track issue density by model and use case Compare issues per passing task, vulnerability density, and code smell density by model so governance teams can spot risky usage patterns.
• Refactor verbose generated functions before production Break up overly long AI-generated methods into smaller units so future review, testing, and incident response remain manageable.

Key takeaways

• GPT-5 can raise benchmark performance while still increasing the burden of code review, refactoring, and quality assurance.
• Complexity and maintainability remain central risk signals even when vulnerability counts look relatively low.
• Practitioners should verify AI-generated code with static analysis and policy gates before production, especially in identity-critical paths.

Key terms

• Code Smell: A code smell is a maintainability problem that makes software harder to understand, test, or change safely. It is not always an immediate exploit, but high smell density increases review burden, slows patching, and raises the chance that security defects survive future changes.
• Cyclomatic Complexity: A measure of how many independent paths exist through a function or module. Higher complexity usually means more branching, more testing effort, and greater risk that a small change will create unexpected behaviour or maintenance overhead.
• Static analysis: Static analysis is the inspection of source code or configuration without executing it. It helps identify insecure patterns early, but its value depends on accuracy, timing, and whether the output is usable enough for developers to fix issues while they are still working on the code.
• Code Verification Loop: A repeating set of checks embedded in development, pull requests, CI, and maintenance to confirm that changes remain safe over time. It treats verification as a living process rather than a one-time approval step.

Deepen your knowledge

NHI Foundation Level course covers NHI governance, workload identity, and secrets management through the industry's only accredited NHI security programme. It helps practitioners connect identity controls to broader software and AI governance decisions.