Notifications
Clear all
Topic starter
20/08/2026 5:51 pm
TL;DR: Among more than 4,000 Java tasks, GPT-5.2 High delivers the strongest security posture at 16 blocker vulnerabilities per MLOC, but also produces the highest code volume at 974,379 LOC, while Claude Sonnet 4.5 reaches 198 blocker vulnerabilities per MLOC and Opus 4.5 Thinking drops that to 44, according to Sonar. Functional correctness is not enough when AI-generated code also creates maintainability and review burden at scale, especially for teams shipping into production.
NHIMG editorial — based on content published by Sonar: LLM code quality analysis and the Sonar LLM Leaderboard
By the numbers:
- Sonar evaluated 4,000+ Java tasks to compare model behaviour across security, reliability, and maintainability metrics.
- Code smells accounted for 92% to 96% of all detected issues across the evaluated models.
Questions worth separating out
Q: How should security teams govern AI-generated code in production environments?
A: Security teams should treat AI-generated code as normal production code with extra provenance risk.
Q: Why do LLMs create security risk even when they pass tests?
A: Because test success only proves the code behaves correctly for the covered cases.
Q: What do security teams get wrong about AI-generated code risk?
A: They often focus on catching insecure output after code is written, which is too late for AI-native workflows.
Practitioner guidance
- Gate AI-generated code with static analysis before merge Require every AI-assisted change to pass static analysis for injection, path handling, concurrency, and resource-leak checks before it reaches review approval.
- Set model-specific risk thresholds for sensitive code paths Use stricter acceptance criteria for authentication, authorization, secrets handling, and other identity-sensitive paths than for low-risk utility code.
- Track maintainability debt as a security metric Measure code smells, complexity growth, and review effort together so teams can see when AI-generated code is creating future remediation cost.
What's in the full report
Sonar's full analysis covers the operational detail this post intentionally leaves for the source:
- Per-model benchmark tables for pass rate, code volume, concurrency defects, and blocker vulnerabilities across the full evaluation set
- Side-by-side comparison data for GPT-5.2 High, GPT-5.1 High, Gemini 3.0 Pro, Opus 4.5 Thinking, and Claude Sonnet 4.5
- The underlying SonarQube static-analysis methodology used to score maintainability and security findings
- The full Sonar LLM Leaderboard view for teams that want to compare code quality signals before selecting a model
👉 Read Sonar's analysis of AI coding model security, complexity, and maintainability →
AI code generation trade-offs: what engineering teams need to know?
Explore further
20/08/2026 6:31 pm
AI code governance now belongs in the same control conversation as application security. The Sonar results show that a model can satisfy test cases while still producing code with security flaws, concurrency defects, and maintainability debt. That means AI-assisted development cannot be governed as a pure productivity initiative. For security leaders, the practical conclusion is that code quality controls need to sit alongside SDLC policy, not after it.
A question worth separating out:
Q: What should teams do when AI-generated code is acceptable functionally but not structurally?
A: Block release until the code meets both functional and structural thresholds. If the logic is too complex, too verbose, or too defect-prone, the team should refactor or regenerate it before it reaches a production branch.
👉 Read our full editorial: AI code quality trade-offs expose security and maintainability debt
