Topic starter
02/08/2026 11:33 am
TL;DR: An independent benchmark against XBOW found 49 verified vulnerabilities versus 31 on two test applications, a 58% gap at the same price tier, with nearly identical false positive rates and fewer than 10% overlapping findings according to Aikido research. The result reinforces that application context, not just scan speed, determines whether AI-assisted pentesting surfaces IDORs, authorization failures, and logic abuse paths.
Editorial analysis by NHI Mgmt Group, based on content published by Aikido: “Aikido vs XBOW: 58% more vulnerabilities found in independent benchmark”.
By the numbers:
- Aikido surfaced 49 verified vulnerabilities on the two applications, while XBOW found 31.
- The benchmark reported a 4% false positive rate for Aikido and a 3% false positive rate for XBOW.
- The two tools agreed on fewer than 10% of vulnerabilities, with 3 matching findings on Fider and 4 on Photoview.
Key questions
Q: What breaks when automated pentests do not understand application context?
A: Without application context, automated pentests tend to find generic issues but miss logic flaws tied to how the business works.
Q: Why do role-based applications need AI pentesting across multiple identities?
A: Because many critical vulnerabilities only appear when one user is compared with another.
Q: How do security teams know if AI pentesting is actually finding the right risks?
A: Look for validated findings in the classes that matter most to your application model, especially authorization issues, object-level access failures, and logic abuse.
Practitioner guidance
- Build role-aware test scenarios Include multiple users, sessions, and object ownership states in every AI-assisted pentest so IDORs and broken authorization can be exercised across boundaries.
- Ingest application context before scanning Feed the agent codebase structure, routes, roles, and business logic before live probing so it can target expected control failures rather than only visible endpoints.
- Separate precision from discovery depth Track false positive rate, unique verified findings, and overlap between tools as different measures so a clean report does not hide shallow coverage.
Bottom line: AI pentesting becomes materially more effective when the agent can interpret application behaviour, not just probe endpoints.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
02/08/2026 12:47 pm
Application context is now a control surface, not a convenience feature. The benchmark shows that AI pentesting quality changes materially when the agent understands the target application's intended behaviour. That means coverage depends on whether the system can reason about roles, objects, and business logic, not just whether it can send more requests. Practitioners should treat context ingestion as part of the control design, because it determines which failures are even discoverable.
A few things that frame the scale:
- The agentic AI market is already worth $5 billion and is projected to reach $50 billion by 2030.
A question worth separating out:
Q: Should organisations prioritise application context or scan speed in AI pentesting?
A: Application context should come first when the goal is to find access-control and logic flaws. Scan speed matters for throughput, but it does not compensate for weak reasoning about roles, objects, and expected behaviour. If the agent cannot understand the application, faster scanning only produces faster incompleteness.
👉 Read our full editorial: AI pentesting benchmark shows context changes vulnerability discovery