Corgea vs Aikido benchmark shows recall matters more than cleaner output

At a glance

What this is: This benchmark compares Corgea and Aikido on the same deliberately vulnerable repository and finds that Corgea delivered far higher recall, catching 42 of 47 confirmed issues.

Why it matters: For AppSec teams, the result underlines a common SAST tradeoff: fewer false positives can still leave too many exploitable findings untouched, which affects remediation prioritisation, developer trust, and backlog completeness.

By the numbers:

• Corgea found 42 of 47 source-confirmed issues, reaching 89.36% recall in the benchmark.
• Aikido found 13 of 47 source-confirmed issues, reaching 27.66% recall in the benchmark.
• F1 score was 85.71%, as 85.71%, compared with Aikido’s 41.94% in the same scoring set.

👉 Read Corgea's benchmark analysis of Corgea vs Aikido on SAST recall

Context

Static analysis tools are often judged on how clean their output looks, but AppSec teams need to know whether the scanner actually finds the issues that matter. In a repository deliberately packed with application, API, Kubernetes, and configuration weaknesses, the meaningful question is coverage, not just precision.

That distinction matters for identity and secrets governance too. Hardcoded credentials, exposed tokens, and insecure deployment templates are classic pathways into non-human identity abuse, and missed findings never reach rotation, revocation, or remediation workflows. On this benchmark, the weaker result was not an edge case; it was the operational risk.

The repository was intentionally noisy and broad, which makes it a useful stress test for custom-code security programmes. Results from such a corpus are not a substitute for your own codebase, but they do expose whether a scanner can handle context, framework behaviour, and cross-file reasoning.

Key questions

Q: What breaks when a SAST tool has low recall?

A: Low recall means confirmed vulnerabilities never make it into triage, so they cannot be assigned, fixed, or verified. In practice, the team may see a cleaner report but still leave exploitable code in production. The real risk is not noise, it is invisible backlog leakage.

Q: Why do context-dependent code flaws matter more than simple injection findings?

A: Context-dependent flaws often determine whether an attacker can cross a trust boundary, bypass authorization, or abuse a framework convention. Simple injection rules catch obvious cases, but they do less well where the issue depends on route handling, logic flow, or multi-file reasoning.

Q: How can teams tell whether their SAST programme is actually working?

A: Look for reduced false-positive churn, faster remediation on high-confidence issues, and consistent developer adoption across repos and teams. If engineers ignore findings, suppress everything by default, or route issues outside the normal delivery process, the programme is not working. Effective SAST changes behaviour, not just dashboards.

Q: Should organisations prioritise recall or precision when choosing a SAST platform?

A: They need both, but the balance depends on programme maturity. Early on, enough recall is essential to expose the real defect population, while precision determines whether developers will keep using the tool. The right choice is the scanner that best fits your review capacity and the kinds of issues your repositories actually contain.

Technical breakdown

Why recall matters more than clean output in SAST

Recall measures how many confirmed issues a scanner finds, while precision measures how many reported findings are real. In code security programmes, recall often determines whether a vulnerability ever enters triage, gets assigned, and reaches a fix. A slightly cleaner report can still be operationally weaker if it omits high-risk flaws that developers never see. This benchmark shows the difference clearly: a tool can look tidier while failing to surface the majority of issues that require action.

Practical implication: score SAST tools on confirmed findings and false negatives, not just on the number of issues they produce.

How framework context changes static analysis coverage

Traditional pattern-based SAST engines are strongest when a vulnerability maps neatly to a known sink, source, or sink-to-source rule. They struggle more when the weakness depends on framework conventions, route decorators, authorization logic, or multi-file data flow. That is why missing authorization on FastAPI routes, SSRF through URL fetches, or open redirects tied to user-controlled parameters often require more than simple regex-style detection. Context and reachability are what separate broad detection from partial pattern matching.

Practical implication: test scanners against framework-specific weaknesses and cross-file flows, not only against basic injection samples.

What the benchmark says about secrets exposure in code

The repository included hardcoded AWS credentials and API tokens, which is a reminder that code scanners also function as secrets controls. Secrets in source code are not just a developer hygiene issue. They create direct paths into cloud resources, deployment systems, and machine identities that inherit trust from those credentials. When a scanner misses exposed secrets, the organisation loses both detection and the chance to revoke or rotate before abuse.

Practical implication: include secret exposure and credential hygiene in every SAST evaluation, especially where code drives cloud or deployment access.

Threat narrative

Attacker objective: The attacker aims to turn code exposure into privileged access, application abuse, or downstream cloud compromise before defenders detect the gap.

1. Entry begins with exposed credentials, hardcoded tokens, or other source-controlled secrets that give an attacker a starting foothold in developer or cloud workflows.
2. Escalation occurs when those credentials map to deployment templates, API access, or privileged automation paths that can be reused beyond their intended scope.
3. Impact follows when missed findings leave exploitable application flaws, secret exposure, or authorization gaps unremediated and available for abuse.

NHI Mgmt Group analysis

Clean output is not the same as complete risk coverage. SAST buyers often overweight precision because noisy findings are painful, but this benchmark shows the real cost sits in false negatives. A scanner that misses 34 confirmed issues may save triage time while leaving the backlog incomplete. For AppSec leaders, completeness is the control objective, not tidiness.

Secrets exposure remains the most identity-relevant failure mode in application security. Hardcoded AWS keys, tokens, and deployment credentials are not merely code defects. They are non-human identities with uncontrolled lifecycle, and they can become the bridge from source code into cloud and CI/CD environments. Practitioners should treat exposed secrets as credential governance failures, not just code hygiene.

Framework-aware reasoning is now part of credible static analysis. Issues such as missing authorization, SSRF, and open redirect logic depend on application context that simple rule engines often under-detect. That does not make pattern matching obsolete, but it does mean modern AppSec programmes need scanners that reason across route handling, trust boundaries, and data flow. Teams should evaluate tools on framework coverage, not just signature coverage.

Named concept: detection-to-remediation gap. This benchmark exposes the distance between finding a vulnerability and placing it into an actionable remediation queue. When detection misses dominate, the organisation does not merely have a scanning problem. It has a governance problem where insecure code can persist without a formal fix path. Security teams should measure how many confirmed issues convert into tickets, owners, and verified closure.

Benchmarking should mirror operational reality, not just demo conditions. The value of this repository is that it mixes obvious injection flaws with contextual issues, exposed secrets, and configuration weaknesses. That is closer to what security teams face in real repositories than narrow toy samples. Buyers should therefore test scanners on repositories that contain both cleanly-patterned defects and logic-heavy weaknesses before deciding how to standardise.

What this signals

Scanner selection is increasingly a governance decision, not just a tooling preference. When a platform misses framework-specific flaws and exposed secrets, the downstream issue is not only weaker detection but a slower control loop across triage, ownership, and remediation.

Detection-to-remediation gap: the measure that matters is how many confirmed issues convert into verified fixes. That is the control lens AppSec teams should apply when evaluating scanners against real repositories, because finding defects without closing them does not reduce exposure.

Identity and secrets controls now sit inside the AppSec evaluation story. A scanner that finds hardcoded credentials can support broader NHI hygiene, but only if the organisation has reliable revocation, rotation, and ownership processes to act on what was found.

For practitioners

• Run scanners against source-confirmed issues Benchmark each candidate on the same repository and classify every result as true positive, false positive, or false negative before comparing vendors.
• Include secrets exposure in every PoC Seed or identify hardcoded credentials, API tokens, and deployment secrets in test repos so the evaluation covers secret discovery as well as code flaws.
• Score framework-aware weaknesses separately Measure detection on missing authorization, SSRF, open redirect, and other context-dependent flaws as a distinct category from injection findings.
• Track remediation conversion, not just findings Follow each confirmed issue through assignment, fix, verification, and closure so the team can see whether detection actually improves the backlog.

Key takeaways

• This benchmark shows that low false positives do not compensate for large numbers of missed confirmed issues.
• The most operationally important gaps were framework-aware flaws and exposed credentials, which are exactly the issues that create remediation and identity risk.
• AppSec teams should evaluate scanners on confirmed findings, false negatives, and fix conversion, not only on how clean the output appears.

Key terms

• False negative: A missed detection where a real threat is not recognised as malicious or important. In AI SOC environments, false negatives often arise when the system under-collects context, stops at the first plausible answer, or over-optimises for speed.
• Recall: The share of real weaknesses that the scanner successfully identifies. High recall reduces the chance of missed vulnerabilities, but if it is achieved by over-reporting, teams can become overwhelmed and stop trusting the results.
• Precision: The share of reported findings that are actually real weaknesses. High precision means fewer false positives, but it does not guarantee that the scanner is finding all of the important problems, so it must be read alongside recall to understand practical effectiveness.
• Secrets Exposure: Secrets exposure is the accidental or uncontrolled disclosure of credentials such as API keys, tokens, certificates, and service passwords. In NHI programs, it matters because a leaked secret often behaves like a live identity, creating immediate access risk until it is revoked or rotated.

What's in the full article

Corgea's full article covers the benchmark detail this post intentionally leaves at the analyst level:

• Side-by-side finding tables showing true positives, false positives, and false negatives for each tool
• Exact examples of the vulnerabilities Corgea found that Aikido missed, including authorization and SSRF cases
• Benchmark scoring method details for precision, recall, and F1 on the 47-issue repository
• Screenshots of the scan outputs and AI Code Audit pricing evidence from the benchmark run

👉 The full Corgea post includes the per-finding breakdown, scoring method, and scan screenshots.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, secrets management, and identity lifecycle control. It is designed for practitioners who need to connect identity governance to application and cloud risk.