Topic starter
11/11/2026 6:04 pm
TL;DR: URL-only black-box DAST solved 51 of 60 Argus challenges, according to ProjectDiscovery. For practitioners, that means evaluation quality now depends on isolation, validation, and budget controls as much as exploit generation.
Editorial analysis by NHI Mgmt Group, based on content published by ProjectDiscovery: “Benchmarking Neo's Black-Box DAST Capabilities”.
By the numbers:
- Neo solved 51 of 60 vulnerable web applications in Argus, which is 85% of the benchmark.
- The average cost per solve was about $3.40.
- The benchmark cap was $20 per challenge.
Key questions
Q: Why do vulnerable web app benchmarks need isolation and reset controls?
A: They need isolation and reset controls so each benchmark starts from a clean application, database and session state, and one test does not poison the next.
Q: What is the difference between SAST and DAST for security teams?
A: SAST inspects code and binaries before execution, while DAST attacks the live application from the outside.
Q: How do black-box DAST agents work against live web apps?
A: Black-box DAST agents interact with a running application using only observable responses, then iteratively test inputs, states, and workflows until they find an exploitable path.
Practitioner guidance
- Implement per-run environment isolation Reset application state, workspace memory, and network reachability for every evaluation so one run cannot influence the next.
- Use exact-value validation for findings Validate flags, proof-of-exploit markers, or other outcomes against the build-time ground truth instead of format checks or pattern matches.
- Pin benchmark dependencies before testing Lock challenge dependencies to known vulnerable versions so patch drift does not erase the flaw you are trying to measure.
Bottom line: URL-only DAST can still reach meaningful web-application coverage, but the result is only trustworthy when the test harness is isolated and exact.
Glossary terms: Black-Box DAST, State Contamination, Dynamic Flag Injection, Multi-Step Exploitation Chain
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
11/11/2026 6:06 pm
URL-only DAST is becoming a governance problem, not just a testing trick. Once an agent can probe live applications without source code, security leaders have to decide whether the output is trustworthy enough to influence remediation priority. That shifts DAST from a simple tool selection issue to a question of evaluation integrity, repeatability, and false-positive control. The control failure is not detection alone, but confidence in what detection actually means.
A few things that frame the scale:
- Internal repositories are 6x more likely to contain secrets than public ones (32.2% vs 5.6%), contradicting the assumption that private repos are safe, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: What should teams do first after a DAST run produces a likely exploit?
A: The first step is to preserve the evidence and contain the runtime state, then validate the finding in a controlled copy of the target before any remediation. Teams should capture the request, response, affected endpoint and session context, while preventing the exploit path from spreading or mutating data during verification.
👉 Read our full editorial: URL-only black-box DAST reached 51 of 60 apps in Argus testing