AI coding agents find real bugs, but noisy results limit trust

At a glance

What this is: Semgrep tested AI coding agents against real Python web apps and found they can uncover genuine bugs, but their findings are noisy and inconsistent.

Why it matters: For AppSec and platform teams, the practical question is not whether AI can find bugs, but where its results can be trusted enough to support triage, review, and prioritisation.

By the numbers:

• Claude Code found 46 vulnerabilities (14% TPR, 86% FPR) across the evaluated applications.
• Codex reported 21 vulnerabilities (18% TPR, 82% FPR) in the same evaluation.
• The researchers reviewed more than 400 security findings produced by the two agents.
• One application produced 3, 6, and 11 distinct findings across three identical runs.

Context

AI coding agents can help surface issues in real code, but vulnerability discovery is only useful if the results can be trusted and reproduced. This article sits in the gap between raw model capability and security operations, where the harder problem is not finding a candidate issue but deciding which findings deserve analyst time.

The identity angle is indirect but real: coding agents are now part of the software delivery and security review chain, which means their output can influence controls around code integrity, AppSec triage, and the governance of AI-assisted development. In practice, the question is whether these agents are acting as helpful assistants or as noisy intermediaries that increase review burden.

The article’s results are typical of early AI-assisted security tooling: promising on some bug classes, weak on semantic reasoning, and unstable across repeated runs.

Key questions

Q: How should security teams handle AI-generated vulnerability findings in the release pipeline?

A: Security teams should treat AI-generated findings as inputs, not decisions. Every finding should pass deterministic validation against source context, runtime evidence, and policy before it influences release gating or remediation. That reduces false confidence and keeps automation from pushing low-quality or environment-agnostic results into production workflows.

Q: Why do AI coding agents produce so many false positives on injection bugs?

A: Injection bugs depend on semantic reasoning across sources, sinks, sanitization, and framework behaviour. AI coding agents can miss one step in that chain and flag safe code as vulnerable, especially when the logic spans multiple files or functions. The result is a high false-positive rate that looks persuasive but fails under verification.

Q: What are the signs that an AI-assisted security review workflow is not reliable enough?

A: The main warning signs are inconsistent findings across repeated runs, a large share of issues that cannot be reproduced, and major performance swings by vulnerability class. If the same prompt produces different issue sets on the same codebase, the workflow is not stable enough to be treated as a primary control.

Q: Should organisations use AI coding agents for vulnerability hunting or human review?

A: Use them for candidate generation and triage assistance, not as a replacement for human review. AI is useful when the task is to surface possible bugs quickly, but humans and deterministic tools are still needed to confirm exploitability, interpret edge cases, and avoid acting on noisy or incorrect results.

Technical breakdown

Why AI coding agents find some bugs but miss others

AI coding agents are strongest when a vulnerability follows a visible pattern in code, such as an obvious object reference or a direct path traversal primitive. They become much less reliable when the issue depends on multi-step reasoning across files, functions, or framework layers. That is why injection-style bugs such as SQL injection and XSS remain difficult: the model must trace data flow, sanitization, framework behaviour, and sink usage correctly. The article shows that contextual pattern matching is not the same as semantic vulnerability analysis.

Practical implication: Use AI findings as leads, not proof, when the bug class depends on inter-procedural data flow or framework-specific semantics.

Why false positives stay high in real application review

A false positive appears when the model flags a code path that is already safe or misreads a harmless pattern as a vulnerability. In real applications, that happens often because the model lacks deterministic state, can mis-handle framework conventions, and may suggest insecure fixes for code that is already correct. The problem is amplified in large codebases where the same pattern can have different meanings depending on surrounding context. That makes raw output hard to operationalise without review discipline.

Practical implication: Route model output through human triage and verification rules before any issue is assigned a remediation ticket.

Why repeated prompts produce different security results

Non-determinism means the same prompt against the same code can produce different findings on different runs. In this evaluation, identical runs on the same application returned different sets of issues, which makes reproducibility and benchmark design especially important. For security teams, non-determinism is not a cosmetic problem. It changes what gets reviewed, what gets escalated, and what gets missed. If a tool cannot reliably reproduce its own output, governance has to compensate for that instability.

Practical implication: Treat reproducibility as a control requirement and benchmark AI-assisted review workflows before relying on them in production.

Threat narrative

Attacker objective: The practical objective is not a direct external exploit but the ability to overwhelm or mislead AppSec workflows with noisy findings that reduce analyst confidence and slow remediation.

1. Entry occurs when the AI coding agent is pointed at a real application codebase and begins generating vulnerability candidates from static context.
2. Credential or logic abuse appears when the agent infers insecure flows, mislabels safe code, or proposes fixes that alter behaviour without understanding the full execution path.
3. Impact is operational rather than exploitative: security teams absorb triage noise, spend time validating false positives, and risk missing real issues hidden inside low-trust output.

Breaches seen in the wild

• Amazon Q Developer extension compromise 2025: An over-scoped CI token let an attacker ship a prompt telling Amazon Q's AI coding agent to wipe files and AWS resources (CVE-2025-8217).
• Sentry MCP Agentjacking 2026: Researchers showed a fake Sentry error, posted with a public DSN, could make AI coding agents run attacker code with developers' credentials.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI coding agents are becoming a security control dependency, not just a productivity feature. Once teams let model output influence bug triage, review prioritisation, or code fixes, the question becomes governance of evidence quality, not novelty. Security teams need to decide where AI assistance ends and where deterministic validation begins. The practitioner conclusion is to treat model output as advisory until it is verified by reproducible controls.

False-positive management is now part of AppSec operating model design. The article shows that AI can find real bugs while still producing too much noise to trust blindly. That creates a workload problem as much as a detection problem, because noisy findings can consume scarce reviewer time and distort risk ranking. The field needs triage rules, confidence thresholds, and validation gates before AI-generated findings enter remediation queues.

Semantic reasoning, not pattern recall, is the real bottleneck. AI agents can recognise vulnerability shapes, but they struggle when the bug depends on control flow, sanitization logic, or multi-file data movement. That limitation matters because injection, authorization, and trust-boundary bugs are exactly where security teams need precision. The practitioner conclusion is to reserve AI for candidate generation and pair it with static analysis and human review for verification.

Non-determinism creates a governance gap in AI-assisted security review. If the same prompt can yield different findings, then the workflow is not auditable in the way most security programmes require. That is especially important for organisations trying to scale AI-assisted AppSec across teams. The practitioner conclusion is to measure repeatability as a control objective, not an afterthought.

AI-assisted vulnerability hunting will work best as a layered system, not a standalone scanner. The article’s results point toward agentic workflows that combine model reasoning with static analysis, test execution, and analyst judgment. That aligns with how modern security programmes already handle uncertain signals: multiple weak indicators become useful only when correlated. The practitioner conclusion is to build systems that amplify verification, not systems that assume the model is the verifier.

From our research library:

• Software supply chain attacks were projected to cost organisations $60 billion in 2025.
• Read next: AI Agent Observability, Audit and Incident Response Guide

What this signals

AI-assisted vulnerability review now needs confidence controls, not just capability claims. The article shows that a model can produce useful leads and still generate enough noise to overload reviewers. For practitioners, that means the governance question is whether the workflow has reproducibility checks, human confirmation gates, and a clear line between candidate discovery and accepted risk.

Context is the scarce resource in AI security review. Models can spot patterns in code, but they struggle when the vulnerability depends on framework conventions, execution order, or sanitization across files. That is why verification layers and deterministic tooling must sit beside AI rather than behind it.

Claude Code-assisted commits leaked secrets at a rate of 3.2%, more than double the human-only baseline of 1.5%, with peaks reaching 31 secrets per 1,000 commits in August 2025. According to the State of Secrets Sprawl 2026, coding agents can amplify security signal volume and security leakage at the same time if governance is weak.

For practitioners

• Establish human verification gates for AI findings Require a reviewer to confirm exploitability, data flow, and framework context before any AI-reported issue becomes a remediation item.
• Benchmark AI scanners by bug class Track performance separately for IDOR, path traversal, SQL injection, SSRF, and XSS so noisy classes do not obscure useful ones.
• Measure repeatability across identical prompts Run the same prompt multiple times on the same codebase and compare issue overlap, severity mix, and missed findings.
• Pair agent output with deterministic analysis Use static analysis, test execution, and code review patterns to validate model claims before making security decisions.

Key takeaways

• AI coding agents can find real vulnerabilities in production-grade code, but their outputs are too noisy to trust without validation.
• The strongest signal in this research is not raw detection power, but the need for reproducible review and human confirmation.
• Security teams should use AI for candidate generation and triage support, then rely on deterministic analysis to confirm exploitability.

Key terms

• False Positive Rate: False positive rate is the share of alerts, detections, or test results that incorrectly indicate a problem when none exists. In security and identity operations, it measures how often a control flags benign activity as suspicious, usually expressed as a percentage of all negative cases. Lower rates reduce analyst noise and wasted response effort.
• True Positive Rate: True positive rate is the share of reported findings that are confirmed as real issues. In security tooling, it measures how often a model or scanner correctly identifies a vulnerability rather than flagging something harmless as suspicious.
• Non-deterministic output: Non-deterministic output means the same prompt or input can produce different responses across runs. In security terms, that makes one-off validation weak because the risk lies in the range of behaviours the system may exhibit, not a single predictable result.
• Interprocedural Taint Analysis: Interprocedural taint analysis tracks untrusted data as it moves across multiple functions and call paths. It helps uncover vulnerabilities that are not obvious in a single file or routine because the risky flow emerges only when code paths are connected. This makes it valuable for deeper application security review.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and agentic AI identity. It helps practitioners connect identity controls to broader security review workflows and governance decisions.