Notifications
Clear all
Topic starter
02/08/2026 11:22 am
TL;DR: Across two open-source applications at the same $4,000 tier, Doyensec’s benchmark found Aikido identified 49 verified vulnerabilities versus 31 for XBOW, according to Aikido. The practical issue is not only detection quality but operational friction, which now shapes how security teams evaluate AI-assisted testing.
NHIMG editorial — based on content published by Aikido: Aikido vs XBOW, an independent benchmark report by Doyensec
By the numbers:
- Doyensec independently benchmarked Aikido and XBOW at the same $4,000 price tier across two real open-source applications, selected at random from 442.
- Aikido found 49 verified vulnerabilities.
Questions worth separating out
Q: How should security teams evaluate AI pentesting tools for enterprise use?
A: Judge them on representative coverage, reproducible proof, and reporting clarity, not on a single benchmark score.
Q: Why does setup friction matter in security testing programmes?
A: Because delay before the first scan delays discovery, reporting, and remediation.
Q: What breaks when retesting is too limited?
A: Remediation confidence drops because teams cannot verify fixes on their normal schedule.
Practitioner guidance
- Benchmark against verified coverage, not raw findings counts Compare tools on unique verified vulnerabilities, severity distribution, and overlap across the same applications before procurement decisions.
- Measure onboarding friction as part of tool evaluation Track time to first scan, number of support exchanges, contract steps, and restarts required before results are produced.
- Align retest terms with remediation cadence Require retesting terms that match how quickly your team closes findings, especially for issues that can expose secrets or access paths.
What's in the full report
Aikido's full benchmark report covers the operational detail this post intentionally leaves for the source:
- Side-by-side evidence tables showing the exact verified findings in Fider and Photoview.
- Workflow detail on the 20-minute setup path, including what was required to start scanning.
- Benchmark notes on retest scope, support overhead, and infrastructure interruptions during the XBOW engagement.
- Full severity and false-positive breakdowns for each tool across the two applications.
👉 Read Aikido's benchmark report on AI pentesting coverage and workflow friction →
AI pentesting coverage: are teams buying outcomes or overhead?
Explore further
02/08/2026 12:40 pm
AI pentesting should be judged as an operational control, not a demo capability. The benchmark shows that output quality alone is not enough to explain tool value. Coverage, turnaround time, and retest accessibility determine whether findings become usable security signal or just another backlog item. For practitioners, that means the real question is how quickly a tool improves risk decisions across the application and identity surfaces it exposes.
A question worth separating out:
Q: What does a benchmark like this reveal about AI-assisted security testing?
A: It shows that real value comes from the full control loop, not just the scan result. Coverage, turnaround time, and retest ease all shape whether findings are useful enough to drive action. Practitioners should treat AI testing as an operating model choice, not a feature checkbox.
👉 Read our full editorial: AI pentesting benchmarks show coverage gaps beyond false positives
