AI-assisted vulnerability detection needs context, not just model scale

At a glance

What this is: This is an analysis of how hybrid static analysis plus AI reasoning can make vulnerability detection more reliable, with context memories and incremental scanning reducing false positives.

Why it matters: It matters because security and IAM teams increasingly need code and policy analysis that is stable enough to trust in production, especially where access logic, secrets, and authorization decisions intersect.

By the numbers:

• Semgrep Multimodal produces true positives results at 37% lower costs compared to finding through LLMs alone.
• Compared to a model-only baseline (Claude Opus 4.6), the hybrid system achieved a 54% reduction in false positive rate.
• 90%, s reduced false positives from the initial approach by 90%, resulting in a false positive rate lower than the model-only baseline.

👉 Read Semgrep's analysis of AI-assisted vulnerability detection with hybrid analysis

Context

AI-assisted vulnerability detection only works in production when it is constrained by code structure, repeatable analysis, and the real design assumptions behind the system. In practice, raw LLM output is too unstable for security workflows unless it is grounded in program analysis and the context that engineers already use to decide what is intended.

The identity angle is indirect but real: authorization logic, service-to-service access, and secrets handling are often encoded in application code, so false positives and inconsistent findings can slow remediation of access flaws. That makes context management a governance issue, not just a model-quality issue. The starting position described here is increasingly typical for teams moving from demos to operational detection.

Semgrep's use of memories and uploaded design context shows a broader pattern for AI security tooling: the model is only useful when it reasons inside the boundaries of a known architecture. That matters for IAM-adjacent code, where trusted exceptions, role mappings, and access assumptions often look suspicious without business context.

Key questions

Q: How should security teams reduce false positives in AI vulnerability scanning?

A: Require scanners to prove reachability, exposure, and privilege context before findings are allowed to block delivery. If a defect cannot be tied to an active runtime path or meaningful data exposure, it should be downgraded or deferred. This keeps engineering attention on exploitable risk and stops alert fatigue from undermining the control.

Q: Why does code context matter so much in AI-powered vulnerability detection?

A: Because code often looks unsafe until you know the intended architecture. Role mappings, trusted profiles, and design assumptions can make a suspicious access path correct, and without that context an LLM will flag it as a flaw. Context turns model output from pattern matching into security judgment.

Q: How do organisations know whether AI-assisted scanning is actually working?

A: Look for stable findings across repeated scans, lower false positive rates, and a shrinking gap between alerts and confirmed issues. If results change unpredictably from run to run, the tool is not operationally trustworthy. Reliability matters as much as raw recall because engineers only fix what they believe.

Q: What should teams do when AI findings conflict with documented design intent?

A: Use the discrepancy as a governance signal, not just a tuning issue. Either the documentation is stale, the intended exception is too broad, or the model lacks enough context to reason correctly. Teams should reconcile the architecture record, update the exception boundary, and re-run the scan on the changed slice.

Technical breakdown

How hybrid static analysis narrows the model's search space

Hybrid detection starts with static analysis, which uses pattern matching plus control-flow and data-flow analysis to identify candidate regions of code that are likely to matter. Instead of asking an LLM to inspect an entire repository, the system slices the program into smaller, relevant segments containing the functions, call paths, sources, sinks, and surrounding logic needed to judge security impact. That reduces token cost, makes the task more deterministic, and lowers the chance that the model will invent missing execution paths. The practical effect is that the AI reasons over evidence, not over noise.

Practical implication: Use static analysis to pre-filter candidate regions before any model-driven review so the AI only evaluates high-signal code paths.

Why context memories change false positive behaviour

AI detection often fails because the model lacks the design assumptions that make a code path intentional. Persistent memories let teams provide scoped instructions, such as whether a backoffice profile is supposed to access sensitive objects, while uploaded threat models and design docs supply architecture context up front. That changes the model's interpretation of what looks risky. The result is not simply fewer alerts, but better alignment between security analysis and how the application was actually designed to work. In governance terms, this is a way to encode approved exceptions without retraining the model.

Practical implication: Capture approved access assumptions and architecture notes as durable context so repeated scans do not rediscover the same intentional exceptions.

Why incremental analysis matters for repeatable scans

Incremental analysis limits re-evaluation to changed code and downstream impact regions. That matters because full-repo rescans can reintroduce stale findings, especially when an AI layer is used on top of static signals. By tracking what changed and where the impact flows, the system preserves consistency across runs and avoids paying for the same noisy evaluation repeatedly. This is especially important in large monorepos, where stable output is often more valuable than marginal recall gains. Determinism is not a luxury in security scanning; it is what makes triage, trend analysis, and engineering trust possible.

Practical implication: Restrict re-analysis to changed regions and downstream dependencies so findings stay stable enough for engineering teams to act on.

NHI Mgmt Group analysis

Context-aware AI detection is really governance-aware detection. Security tooling that cannot absorb design assumptions will keep misclassifying intended access paths as flaws. In codebases where role logic, service permissions, and secrets handling are intertwined, the quality of the context is as important as the quality of the model. Practitioners should treat context ingestion as a control surface, not a convenience feature.

Determinism is a security requirement, not a product preference. If the same scan produces different findings from run to run, engineers will not trust the pipeline and remediation will stall. The combination of static analysis and constrained AI reasoning is therefore more than an accuracy improvement. It is a way to make security findings operationally repeatable, which is essential for any programme that relies on code scanning at scale.

AI-assisted code review creates an access-governance problem whenever the code expresses authorisation logic. When the article's subject includes sensitive object access and trusted exceptions, the real issue is not just detection quality. It is whether teams can document, preserve, and audit the assumptions that separate intentional access from exploitable access. That maps directly to NHI and IAM governance in application-layer policy paths.

False-positive reduction only matters if it shortens the time from detection to remediation. The article's core claim is that smaller, better-scoped evaluation sets improve trust and coverage at the same time. For teams running application security, the practitioner conclusion is to align AI-assisted review with reviewable code slices, documented controls, and ownership for exceptions.

AI governance debt: the accumulation of undocumented design assumptions, stale exceptions, and unstable scan output that makes model-assisted security tools hard to trust. Once that debt builds up, teams spend more time arguing with findings than fixing code. The way out is to bind AI reasoning to architecture evidence and controlled scope.

What this signals

AI governance debt: teams that let undocumented exceptions accumulate will end up tuning around noise instead of reducing risk. The better operating model is to treat context, scan scope, and exception handling as part of the control framework, then pair that with reference guidance such as the NIST Cybersecurity Framework 2.0 and the NIST SP 800-53 Rev 5 Security and Privacy Controls.

For identity-heavy code paths, the practical signal is whether your scan pipeline can explain trusted access without burying engineers in false positives. That is where the combination of program structure, documented assumptions, and lifecycle discipline matters most, and where the NHI Lifecycle Management Guide is a useful reference point.

As AI becomes part of vulnerability triage, the programme question is not whether the model is clever enough, but whether the surrounding process is stable enough to support remediation. If findings are inconsistent, costlier to run, or detached from architecture reality, the issue is governance maturity rather than model capability.

For practitioners

• Define a context intake standard for code scanning Require threat models, architecture notes, and approved access assumptions to be attached to security scans so AI findings reflect intended behaviour, not guesswork.
• Scope AI evaluation to high-signal code regions Use static analysis, control flow, and data flow to narrow model review to candidate paths instead of scanning entire repositories with the LLM.
• Treat approved exceptions as auditable memory Maintain persistent, scoped context for intentional access patterns so repeated scans do not recreate the same false positives across releases.
• Measure finding stability across repeated scans Track whether the same code produces the same findings over multiple runs, because variability is a governance defect when engineering teams depend on consistent triage.
• Restrict rescans to changed code and downstream impact Use incremental analysis so only changed regions and their dependency paths are re-evaluated, which reduces stale findings and wasted model spend.

Key takeaways

• AI-assisted vulnerability detection becomes reliable only when static analysis, context, and constrained model reasoning work together.
• False positives are a governance problem as much as a technical one because unstable findings erode engineering trust and slow remediation.
• Teams that want production-grade AI scanning should measure repeatability, exception handling, and scan scope before they measure model sophistication.

Key terms

• Hybrid vulnerability detection: A detection approach that combines deterministic static analysis with AI reasoning. Static methods narrow the search space, while the model interprets context and intent in the code slice that matters. The goal is better precision, coverage, and operational trust than either method usually provides alone.
• Context Memory: Context memory is the structured information an AI agent uses to interpret alerts, assets, identities, and prior decisions. When poorly maintained, it can cause confident but wrong investigations. When governed well, it helps the agent reason consistently within the organisation’s operational reality.
• Incremental analysis: A scanning method that re-evaluates only changed code and the downstream regions affected by those changes. It avoids full repository rescans, cuts compute cost, and reduces stale findings. For production workflows, it is a key mechanism for keeping AI-assisted review stable and affordable.
• AI Governance: AI governance is the set of controls used to discover, classify, approve, restrict, monitor, and revoke AI-enabled access. It connects identity, data, and policy so organisations can manage what AI can reach, what it can share, and when it should be stopped.

What's in the full article

Semgrep's full post covers the implementation detail this analysis intentionally leaves at the architectural level:

• How Semgrep structures candidate generation with control flow and data flow analysis for large codebases
• How its memories and uploaded context are scoped and applied during detection and triage
• How incremental analysis limits rescans to changed regions and downstream impact paths
• How the hybrid approach compares operationally on repeated scans and false positive reduction

👉 Semgrep's full post covers the candidate generation, context memories, and incremental analysis approach in more implementation detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps security practitioners connect identity controls to the broader security programmes they operate.