Corgea vs Snyk on a vulnerable repo: what the benchmark shows

At a glance

What this is: This benchmark compares Corgea and Snyk on the same deliberately vulnerable repository and finds Corgea detected more confirmed issues with better overall scoring.

Why it matters: For practitioners, the result matters because missed code, configuration, and authorization flaws become untracked backlog items, while false positives dilute response capacity and slow remediation.

By the numbers:

• Corgea found 42 of 47 source-confirmed issues and Snyk found 26 in the same benchmark set.
• Corgea reached 89.36% recall and 85.71% F1, while Snyk reached 55.32% recall and 65.00% F1.
• Corgea reported 82.35% precision versus Snyk's 78.79% precision on the same scoring set.

👉 Read Corgea's benchmark analysis of Corgea vs Snyk on vulnerable code

Context

Static analysis only helps when it consistently surfaces the weaknesses that matter most in real code: missing authorization, unsafe parser settings, exposed credentials, and vulnerable request handling. In this benchmark, the question is not whether either tool can find obvious bugs, but which one produces broader, more actionable coverage across a mixed application and configuration repository.

For IAM and identity-adjacent teams, the governance angle is straightforward. Access control failures in application code often sit alongside secret exposure, deployment misconfigurations, and trust-boundary breaks, so scanner performance affects both application security and identity hygiene. That makes SAST evaluation a control-quality issue, not just a tooling preference.

Key questions

Q: How should teams evaluate SAST tools in mixed application and configuration repositories?

A: Use a fixed benchmark that includes application code, infrastructure templates, and identity-related control failures. Score the tool on confirmed findings, missed issues, and reviewer-accepted precision, because a scanner that cannot cross those boundaries will not reflect the real attack surface. The best evaluation mirrors your own code paths, not a vendor demo repository.

Q: Why do false negatives matter more than high alert volume in SAST?

A: False negatives leave exploitable issues outside the backlog, so nothing gets triaged or remediated. In practice, one missed authorization flaw or exposed credential can matter more than dozens of noisy alerts because the missing finding never enters the control process. Coverage is therefore a direct input to risk reduction.

Q: What do security teams get wrong when comparing pentesting tools?

A: They often compare output volume, interface polish, or feature lists instead of asking how the platform validates findings and limits unsafe access. A useful comparison starts with the quality of evidence, the scope of inspection, and the strength of guardrails around sensitive code and credentials.

Q: Should organisations prioritise recall or precision when choosing a SAST platform?

A: They need both, but the balance depends on programme maturity. Early on, enough recall is essential to expose the real defect population, while precision determines whether developers will keep using the tool. The right choice is the scanner that best fits your review capacity and the kinds of issues your repositories actually contain.

Technical breakdown

Why coverage gaps matter in mixed code and configuration repositories

SAST tools do more than identify source code vulnerabilities. In a repository that combines application logic, infrastructure templates, and deployment configuration, the scanner has to reason across multiple trust boundaries at once. A tool that is strong on request-to-sink taint flow can still miss authentication gaps, exposed credentials, or parser misconfiguration if those issues sit outside its narrow pattern library. That is why recall matters as much as precision: unreported findings never reach triage, remediation, or risk reporting.

Practical implication: benchmark scanners against the code and configuration patterns you actually run, not only against a few obvious injection cases.

How false positives distort remediation capacity

A false positive is not just an incorrect alert. It consumes analyst time, creates friction with developers, and weakens trust in the scanning programme. In practice, repeated low-quality findings push teams toward alert suppression or shallow review, which can hide genuine issues in later scans. Precision therefore matters operationally because it governs whether the backlog reflects reality or scanner noise. The most useful scanners maintain enough precision to keep review credible while still surfacing the long-tail issues that matter.

Practical implication: measure the percent of findings that survive code review and use that number to decide whether the scanner is fit for developer-facing workflows.

What AI-native SAST changes in the evaluation model

AI-native analysis is often strongest when vulnerability context spans multiple files, framework conventions, or non-obvious authorization boundaries. Traditional pattern matching can catch canonical issues such as SQL injection, but it may underperform on application-specific misuse like unauthorised data-modifying routes, credential exposure in deployment manifests, or parser behaviour that depends on runtime configuration. The architectural shift is from finding a known syntax pattern to understanding whether a risky behaviour is reachable and meaningful in the application context.

Practical implication: include multi-file authorization and configuration scenarios in your tool evaluations, not just single-function sink tests.

Threat narrative

Attacker objective: The practical objective is to keep exploitable application and configuration weaknesses unremediated long enough to enable unauthorised access, data exposure, or code execution.

1. Entry occurs when an attacker or reviewer-visible flaw is present in application code, deployment templates, or configuration files and the scanner must recognise it as a real weakness.
2. Escalation follows when the weakness enables unauthorised access, parser abuse, command execution, or credential exposure that expands the blast radius beyond the initial code path.
3. Impact is the security team missing confirmed issues or wasting capacity on false positives, which delays remediation and preserves exploitable conditions in production.

NHI Mgmt Group analysis

Coverage, not just signal quality, is the real SAST differentiator. A scanner that finds only the most obvious injection paths leaves authentication, configuration, and authorization weaknesses outside the security backlog. In mixed repositories, that is a governance failure because the organisation cannot remediate what it never sees. Practitioners should treat recall as a control objective, not a reporting metric.

Precision still determines whether developer teams trust the programme. Findings that duplicate existing alerts, point to the wrong line, or misclassify a condition as a vulnerability create review drag and erode adoption. The operational threshold is not perfection, but enough accuracy that engineers will keep engaging with the results. Security teams should therefore evaluate precision against reviewer acceptance, not only against vendor scorecards.

Context-aware analysis is becoming the expected baseline for modern SAST. The benchmark shows why application security programmes are moving beyond single-pattern detection toward reasoning about framework behaviour, deployment configuration, and multi-file reachability. That shift aligns with NIST SP 800-53 Rev 5 security and privacy controls and broader NIST CSF expectations for continuous control effectiveness. Teams should expect scanners to explain why a finding is reachable, not just where a pattern appears.

Identity and application security are converging at the code layer. The most consequential misses in this benchmark include unauthorised data-modifying routes and credential exposure in deployment manifests, both of which sit at the boundary between software behaviour and access governance. That means IAM and application security teams need shared criteria for what counts as an access control failure in code. Practitioners should align scanner output with identity risk, not treat it as a separate AppSec problem.

Confirmed-finding benchmarks are more useful than marketing comparisons. Comparing tools on the same repository, same issue set, and same review rules exposes the practical gap between reported detections and security value. That is the right standard for procurement and programme tuning because it measures what survives human validation. Teams should insist on benchmark methods that mirror their own review workflow and defect types.

What this signals

Coverage debt: security teams are increasingly judged on whether their scanners can find the things that matter, not whether they can generate large result sets. When application, configuration, and identity failures coexist in the same repo, the control model has to account for all three or the backlog becomes misleading. Practitioners should expect benchmark-style evidence to carry more weight in procurement and tuning decisions.

From our research, leaked-secret remediation remains slow enough to preserve exposure windows that overlap with CI/CD and runtime usage. The average estimated time to remediate a leaked secret is 27 days, which means scanner misses and secret exposure problems can compound for weeks rather than hours. That makes evaluation of detection quality and remediation workflow a governance issue, not just an engineering metric.

Scanner selection is moving toward control validation. The teams that will get better outcomes are those that evaluate whether a tool can find unauthorised access paths, exposed credentials, and configuration-driven weaknesses in the same pass, then use that output to prioritise review capacity.

For practitioners

• Benchmark scanners against your own repository mix Use a fixed issue set that includes application logic, deployment templates, parser configuration, and identity-related control failures. Compare confirmed findings, not raw alert counts, so you can see what will actually enter remediation.
• Measure reviewer-accepted precision Track the percentage of findings that survive engineer review without reclassification or dismissal. If precision drops below a level your teams will tolerate, the tool will not scale in a developer workflow.
• Test for multi-file authorization and config issues Include unauthorised write paths, credential exposure in manifests, and unsafe parser settings in your evaluation rubric. These are the classes most often missed when a scanner is only optimised for classic source-to-sink patterns.
• Separate duplicate alerts from new coverage Count whether a finding adds a new exploitable path or merely restates a known issue under a different label. Duplicate framing inflates tool value without improving remediation coverage.

Key takeaways

• This benchmark suggests that scanner quality is defined by confirmed coverage across code and configuration, not by raw alert volume.
• Corgea's advantage came from surfacing more real issues and missing fewer benchmark findings, while Snyk retained value on several issue types Corgea missed.
• Practitioners should evaluate SAST with the same discipline they apply to access controls, because unreported flaws and noisy alerts both weaken governance.

Key terms

• Static Application Security Testing: Static Application Security Testing is a method for finding security flaws by examining code, binaries, or configuration without executing the application. It is strongest when used early in development, where teams can fix issues before deployment and prevent avoidable defects from reaching production.
• False Positive: A false positive is a scanner result that looks like a secret but is not actually sensitive. In secret governance, false positives matter because they consume analyst time, weaken trust in alerts, and can delay response to the findings that truly change exposure and access risk.
• False negative: A missed detection where a real threat is not recognised as malicious or important. In AI SOC environments, false negatives often arise when the system under-collects context, stops at the first plausible answer, or over-optimises for speed.
• Recall: The share of real weaknesses that the scanner successfully identifies. High recall reduces the chance of missed vulnerabilities, but if it is achieved by over-reporting, teams can become overwhelmed and stop trusting the results.

What's in the full report

Corgea's full benchmark analysis covers the operational detail this post intentionally leaves for the source:

• The full issue-by-issue classification table showing true positives, false positives, and false negatives for both tools.
• Repository-specific examples of the findings Corgea missed and the findings Snyk missed, useful for control tuning.
• The reviewer methodology used to classify findings against source-confirmed issues in the benchmark set.
• The supplemental observations on Snyk-only rows that were discussed separately from the fixed benchmark denominator.

👉 The full Corgea benchmark review includes per-finding classifications, missed issues, and methodology details.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, IAM, and secrets management. It is designed for practitioners who need to connect identity controls to broader security programmes and operational risk.