AI-generated code needs hybrid AppSec controls, not model-only security

At a glance

What this is: Checkmarx argues that AI-generated code still needs hybrid AppSec controls because model output quality does not reliably translate into secure code.

Why it matters: For IAM, NHI, and broader security teams, the lesson is that AI-assisted software development introduces governance gaps where verification, remediation, and inventory controls must be deterministic, not model-dependent.

By the numbers:

• Frontier models produced working code 83% to 95% of the time, but secure and functional output still reached only 24% to 36%.
• Adding security steps raised secure and functional output from 24% to 36% to 47% to 56%.

Context

AI-generated code creates a governance problem when the system that writes code is treated as the same control that validates it. Functional output and secure output are not the same thing, so application security teams need separate mechanisms for generation, verification, and remediation.

That distinction matters for identity and AI governance as much as for AppSec. When codebases increasingly contain models, agents, MCP servers, and other machine components, teams need to know what exists, what it can reach, and which controls can independently verify its behaviour.

Key questions

Q: How should AppSec teams govern AI-generated code without relying on the model to validate itself?

A: Use independent deterministic controls for verification and keep them separate from the system that produced the code. That preserves auditability, reduces conflict of interest, and gives teams repeatable security decisions even when generation is probabilistic. The model can assist with context and remediation, but it should not be the sole judge of its own output.

Q: Why does functional code quality not guarantee secure code in AI-assisted development?

A: Because code can run correctly while still containing insecure logic, weak authorization, unsafe assumptions, or unreviewed dependencies. Functional correctness measures whether the task was completed, while security measures whether the implementation resists abuse. In AI-assisted workflows, those are separate outcomes and must be measured separately.

Q: What are the signs that autonomous coding is outpacing security review capacity?

A: Common signs include a sharp rise in AI-generated pull requests, longer review queues, rushed or shallow reviews, and a growing dependence on review bots to compensate for human overload. If teams start merging more quickly without stronger controls, that usually signals the process has shifted faster than governance, testing, and assurance can keep up.

Q: Should organisations compare AI code generation speed with AppSec remediation speed?

A: Yes, because the real risk is not generation alone but the mismatch between how quickly code is produced and how quickly vulnerabilities are resolved. If remediation cannot keep pace, AI simply increases the volume of insecure changes entering the pipeline. Teams should measure both throughput and fix velocity as linked governance signals.

Technical breakdown

Why functional code is not secure code

Large language models can generate code that compiles, runs, and satisfies the immediate task, but that does not mean the code is safe to ship. Security failures often arise from missing validation, unsafe assumptions, weak authorization paths, or inputs that are not constrained by policy. A model optimised for completion is not inherently optimised for secure implementation. That is why benchmark success rates in generated code can look strong while secure pass rates remain much lower. In practice, the gap is structural: generation is probabilistic, while security enforcement must be deterministic.

Practical implication: Treat model output as an input to AppSec controls, not as evidence that the code is safe.

Why hybrid AppSec beats model-only review

A hybrid AppSec approach combines deterministic scanning and policy enforcement with AI-assisted reasoning for context and remediation. Deterministic tools create repeatable findings and auditability, while probabilistic systems can help identify patterns, rank findings, and propose fixes. The problem with model-only review is conflict of interest: the same system that produced the code is being asked to judge its own output. That weakens assurance, especially when teams need traceable decisions, consistent enforcement, and evidence they can defend in audits or incident reviews.

Practical implication: Separate code generation from independent security verification so one system is not validating its own output.

Why AI inventory becomes part of AppSec governance

AI-generated code introduces new assets into the software estate, including models, agents, SDKs, and MCP servers. Once those components are embedded in applications, teams need an inventory that is continuous and policy-aware, not just a static list for documentation. AI inventory is therefore a governance control as much as a discovery control: it supports auditability, ownership, and risk management across the development lifecycle. Without that visibility, organisations can secure the code they see while missing the AI components that shape behaviour behind it.

Practical implication: Maintain a living inventory of AI components so security review includes the systems influencing code behaviour.

NHI Mgmt Group analysis

Model quality is not a control. High functional output from frontier models does not close the AppSec gap because secure code requires enforcement, not prediction. The article’s core finding is that code generation and code assurance are different disciplines, and organisations that blur them will underinvest in verification. The practitioner conclusion is simple: security gates must remain independent of the model that produced the change.

Hybrid AppSec is a governance model, not a tooling bundle. The value of combining deterministic and probabilistic methods is not speed alone, but defensible decision-making across review, triage, and remediation. That matters wherever AI now influences software delivery, because assurance has to survive audits, incidents, and developer turnover. Practitioners should treat the hybrid pattern as a control architecture, not a feature comparison.

AI inventory is the missing control plane for AI-in-code environments. When applications contain models, agents, MCP servers, and SDKs, teams need asset visibility before they can assign ownership or assess risk. This is where application security intersects with AI governance and, indirectly, identity governance, because unmanaged AI components behave like ungoverned non-human actors. The practitioner conclusion is to inventory first, then enforce policy.

Security debt shifts left when remediation does not. The article shows that discovery alone is no longer sufficient when AI increases code volume and velocity. What matters is whether teams can convert findings into merge-ready fixes before defects multiply across the pipeline. That makes remediation capacity, not just detection capacity, a central measure of AppSec maturity.

AI governance debt: Organisations that deploy AI-assisted development without inventory, verification, and remediation controls are accumulating a governance problem they will later have to unwind. The issue is not the model itself but the absence of independent controls around it. Practitioners should frame AI coding adoption as a control design exercise, not a productivity experiment.

From our research library:

• The average time to mitigate a leaked secret is 36 hours, highlighting the operational burden of manual remediation processes, according to the 2024 State of Secrets Management Survey.
• The average estimated time to remediate a leaked secret is 27 days, despite 75% of organisations expressing strong confidence in their secrets management capabilities, according to the State of Secrets in AppSec.
• Read next: Identity and NHI Security Business Case Guide

What this signals

AI governance debt: AI-assisted development creates a control problem when inventory, verification, and remediation are treated as optional layers instead of core governance. Organisations that scale code generation faster than their security review process will accumulate risk in the backlog, not just in the runtime estate.

The more AI influences software delivery, the more important it becomes to know which components exist and who can change them. That is where application security starts to overlap with identity governance, because models, agents, and MCP servers behave like non-human actors that require ownership, policy boundaries, and traceability.

For practitioners

• Separate generation from verification Keep deterministic security checks independent from the model that wrote the code so review remains auditable and repeatable.
• Shift security earlier in the SDLC Move vulnerability detection into the IDE and build stages so fixes happen before flaws propagate into runtime and backlog.
• Measure remediation, not just detection Track how quickly findings become merge-ready fixes, because AI-driven code volume only helps if the organisation can close issues at the same pace.
• Inventory AI components in code Maintain continuous visibility into models, agents, MCP servers, and SDKs so policy can be applied to the actual runtime estate.

Key takeaways

• AI-generated code can be functional without being secure, so generation quality alone is not a valid security control.
• Hybrid AppSec works because deterministic verification and AI-assisted remediation compensate for the limits of model-only review.
• Continuous inventory of AI components is now part of application governance, not just documentation hygiene.

Key terms

• Hybrid AppSec Model: A hybrid AppSec model combines probabilistic AI reasoning with deterministic validation. The AI layer helps discover patterns quickly, while the deterministic layer confirms what is real and actionable, reducing noise without giving up speed or coverage.
• Deterministic Verification: A verification method that produces the same enforced result every time when the required proof is present, instead of relying on human judgement or probabilistic signals. It is useful for high-blast-radius access changes because it removes discretion from the decision point.
• AI Inventory: An AI inventory is a governed record of all AI-related assets, enriched with owner, purpose, access, and risk context. It turns discovery into something security, compliance, and IAM teams can use to make approval, review, and revocation decisions.
• API Governance Debt: API governance debt is the accumulation of weak, inconsistent, or missing controls around how APIs are designed, approved, documented, secured, and retired. It creates hidden risk over time. Technically, it includes gaps in authentication, authorization, versioning, inventory, ownership, logging, and policy enforcement that make APIs harder to trust and govern.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, machine identity security, and secrets management. It helps practitioners connect identity control design to the broader security programmes they run.