Topic starter
02/08/2026 11:22 am
TL;DR: The benchmark found Aikido identified 49 verified vulnerabilities versus 31 for XBOW at the same $4,000 tier, with Doyensec pointing to a gap between noise reduction and discovery breadth. Security teams should weigh verified coverage and retest burden when comparing AI-assisted testing tools.
Editorial analysis by NHI Mgmt Group, based on content published by Aikido: “Aikido vs XBOW - Independent benchmark report by Doyensec”.
By the numbers:
- Aikido found 49 verified vulnerabilities, compared with 31 for XBOW, across two open-source applications at the same $4,000 tier.
Key questions
Q: Why do support overhead and scan restarts matter in AI pentesting?
A: Because they change how often a team can test, retest, and operationalise results.
Q: What do security teams get wrong about AI pentesting vendor claims?
A: They often focus on feature breadth instead of operational proof.
Q: What should teams do when an AI pentesting tool has good precision but thin coverage?
A: Treat precision as only one input and compare verified findings across the same targets before standardising on a tool.
Practitioner guidance
- Benchmark validated coverage on the same targets Run AI pentesting candidates against identical applications and require manually validated findings before comparing results.
- Measure operational friction as a control metric Track setup time, support interactions, scan restarts, and retest turnaround alongside discovery results.
- Separate precision from breadth in tool evaluation Review false-positive rate, severity accuracy, and total verified findings as distinct measures.
Bottom line: The benchmark points to a gap between noise reduction and discovery breadth in AI pentesting tools.
Glossary terms: Verified vulnerability, Coverage Breadth, Retest Cadence, Operational Friction
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
This topic was modified 2 days ago by NHI Mgmt Group
02/08/2026 12:40 pm
Coverage is the real control variable in AI-assisted testing. False-positive rates are useful, but they do not tell you whether the tool is seeing enough of the attack surface to matter. In this benchmark, the gap between 49 and 31 verified findings suggests that practitioners should focus on validated discovery breadth, not just output cleanliness. The practitioner conclusion is simple: if coverage is thin, the tool may be under-delivering even when the reports look tidy.
A question worth separating out:
Q: What does a benchmark with limited overlap between tools actually tell practitioners?
A: It usually means the products are exploring different parts of the attack surface, so one may be missing classes of issues the other finds. In that situation, buyers should ask which coverage profile matches their risk model, not which report looks cleaner.
👉 Read our full editorial: AI pentesting benchmarks show coverage gaps beyond false positives