AI-assisted vulnerability detection needs context, not just model scale

At a glance

What this is: Semgrep argues that AI-assisted vulnerability detection only works reliably when model reasoning is constrained by static analysis and codebase context.

Why it matters: This matters to security teams because false positives, unstable findings, and uncontrolled LLM cost make AI detection unusable unless governance context and program structure are built into the analysis path.

By the numbers:

• Semgrep Multimodal produced true positives at 37% lower costs compared with finding through LLMs alone.
• The hybrid system reduced the false positive rate by 54% versus the model-only baseline.
• In one phase, the approach reduced false positives by 90% from the initial hybrid system.

Context

AI-assisted vulnerability detection is not the same problem as vulnerability detection itself. The challenge is turning probabilistic model output into stable security analysis that engineers can trust, especially when scanning large codebases with complex control flow and data flow. In practice, the governance gap is not model intelligence alone but whether the system can preserve context, determinism, and explainability across repeated scans.

For security and identity practitioners, the most relevant intersection is not human identity but the identity of code paths, trust assumptions, and privileged behaviours inside software systems. When an AI model reasons without those boundaries, it can misread intentional access patterns as flaws or miss the execution context that determines whether a pattern is exploitable. That makes code context a control surface, not a convenience.

Semgrep's approach places static analysis in front of AI reasoning so the model evaluates only targeted program slices. That makes the detection pipeline closer to a governed analysis workflow than a free-form prompt over raw source, which is a more realistic model for production security operations.

Key questions

Q: Why do AI vulnerability scanners produce so many false positives without code context?

A: Because the model cannot reliably tell whether a pattern is intentional or exploitable when it sees code without the surrounding execution path, design assumption, or trust boundary. Static analysis and context documents reduce that ambiguity by bounding the reasoning problem before the model decides whether a finding is real.

Q: How should teams implement AI-assisted vulnerability detection in large codebases?

A: Start with static analysis to generate security-relevant candidates, then pass only focused program slices to the model. That sequence keeps costs manageable, improves repeatability, and gives reviewers findings that are tied to actual control flow and data flow instead of isolated snippets.

Q: What signs show that an AI code scanner is not trustworthy enough for production use?

A: Repeated scans produce different results on unchanged code, findings disappear and reappear without code changes, and analysts spend most of their time dismissing noise. Those symptoms show the tool is missing structural context and is not stable enough to support operational triage.

Q: Should security teams prioritise context management or model scale for AI detection?

A: Context management should come first, because larger models do not solve missing design assumptions, unstable output, or poor precision. A smaller model guided by deterministic analysis and trusted context is usually more useful than a larger model scanning raw code.

Technical breakdown

Why raw LLM scanning of code produces noisy findings

Large language models are probabilistic, so their output changes when the same code is presented in different ways or with different surrounding text. When they are asked to inspect entire files without structural guidance, they often lose the execution context needed to distinguish an actual vulnerability from an intentional design choice. That is why raw LLM scanning tends to inflate false positives and produce unstable findings. The core technical issue is not just model quality. It is the absence of a reliable boundary around the code region that actually matters.

Practical implication: Use code structure and data flow to narrow the model’s scope before trusting any AI-generated finding.

How static analysis constrains AI vulnerability detection

Static analysis identifies candidate regions using pattern matching, control flow, and data flow so the model is only asked to reason about code that already looks security-relevant. The system then builds slices that include the relevant functions, call paths, sources, sinks, and nearby conditional logic. That gives the model enough context to judge intent and exploitability without reading the entire repository. The result is a deterministic front end with a probabilistic reasoning layer on top, which is a much better fit for production detection than end-to-end model inference.

Practical implication: Anchor AI review to candidate generation and sliced program context instead of sending raw repositories to a model.

Why context memories and documents change the security signal

A memory in this setting is persistent user or organisation guidance that tells the model how to interpret a codebase. Context documents do the same thing at a broader level by supplying threat models, design assumptions, and security decisions already made by the team. This matters because some code patterns are only risky if those assumptions are missing. If the model does not know an access path is intentionally permitted, it will treat it as suspicious and pollute triage. Context turns the scan from generic pattern matching into analysis against the system’s actual operating assumptions.

Practical implication: Feed approved design assumptions into analysis so intentional behaviours do not keep resurfacing as false positives.

NHI Mgmt Group analysis

AI-assisted code scanning fails when context is treated as optional: model scale alone does not create trustworthy vulnerability detection. A scanner needs the system’s design assumptions, execution paths, and explicit trust boundaries or it will over-report intentional behaviour as defects. The practical conclusion is that context management is a security control, not just a tuning input.

Deterministic candidate generation is the real governance layer in hybrid detection: the model should not be the first line of reasoning over a codebase. Static analysis, control flow, and data flow are the mechanisms that make AI output repeatable enough for engineering teams to use. That pattern matters because trust in security tooling collapses when the same code produces inconsistent results.

Code context is the missing control for modern software assurance: the article shows that security analysis becomes more operational when the model works inside a bounded slice of the program rather than the whole repository. That is especially relevant for teams governing complex codebases, where the question is not whether AI can find issues but whether it can do so without overwhelming reviewers. Practitioners should treat context as part of the detection architecture, not an afterthought.

37% lower cost is only meaningful because precision and repeatability improved too: cost reduction by itself would be a weak metric if the scan quality fell. The article's real signal is that constrained reasoning can improve both economics and trust at the same time. That is the direction the security market is heading, where AI must be governed by structural analysis to be operationally credible.

Hybrid analysis is the more defensible model for AI in security operations: it keeps semantic reasoning where it adds value and uses deterministic analysis where reliability matters most. That division of labour is the right pattern for security tooling that must support evidence-based triage. Practitioners should expect the strongest systems to combine machine reasoning with explicit program context, not replace one with the other.

What this signals

Context-constrained inference is the real control plane for AI-assisted detection: security teams should assume that raw model output is too variable to govern on its own. The operating model that scales is one where deterministic analysis defines the boundary and AI only reasons inside it.

Program slicing will matter more than prompt tuning: if the scanner cannot isolate sources, sinks, and relevant call paths, it will keep paying the cost of broad inference while still missing the engineering intent that determines exploitability. That shifts investment toward analysis architecture, not just model selection.

For governance teams, the key question is repeatability: if unchanged code does not yield unchanged findings, the detector is still experimental from an assurance perspective. The target is a workflow that produces stable, reviewable evidence rather than impressive but ungoverned outputs.

For practitioners

• Define approved codebase context for AI detection Capture threat models, design assumptions, and known trusted access patterns as persistent context so the scanner can distinguish intended behaviour from suspicious behaviour.
• Gate model reasoning behind static candidate generation Require control flow and data flow analysis to narrow the regions the model reviews, especially in large repositories and monorepos where raw inference is too broad.
• Measure stability across repeated scans Track whether the same code produces the same findings over time, then investigate variance as a tooling problem rather than assuming the code changed.
• Use incremental analysis to limit rework Restrict re-evaluation to changed regions and downstream impact paths so unchanged code that already created noise does not keep resurfacing.

Key takeaways

• AI-assisted vulnerability detection fails when models are asked to judge raw code without the structural and governance context that determines whether a pattern is real.
• Semgrep's hybrid approach improved reliability by combining static analysis, user-provided context, and targeted model reasoning, while also lowering cost and false positives.
• For practitioners, the operational lesson is to treat context, candidate generation, and repeatability as core security controls rather than optional tuning choices.

Key terms

• Program Slice: A program slice is the smallest relevant subset of code needed to understand a specific behaviour or security question. It includes the functions, call paths, and data flow paths that materially affect the outcome, which makes automated reasoning more precise and less noisy.
• False Positive: A false positive is a scanner result that looks like a secret but is not actually sensitive. In secret governance, false positives matter because they consume analyst time, weaken trust in alerts, and can delay response to the findings that truly change exposure and access risk.
• Deterministic program analysis: Deterministic program analysis is code analysis that identifies relevant paths, sinks, and candidate locations through exact rules rather than probabilistic guessing. In a mixed AI workflow, it acts as the coverage engine, feeding the model the code slices it should reason about instead of asking the model to search the whole repository.
• Code Context: Code context is the surrounding information that explains how a software issue behaves in the application. It includes control flow, component interaction, and developer intent, which can change whether a finding is exploitable, low risk, or urgent. Context helps security teams distinguish real risk from theoretical weakness.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. Explore it if your programme needs a tighter link between governance assumptions and real-world control decisions.