Topic starter
18/08/2026 2:42 pm
TL;DR: AFL++ campaigns can generate billions of executions and thousands of crashes while still yielding only a handful of real bugs, according to Appknox, because crash volume, coverage, and execution speed do not equal vulnerability discovery. The lesson is that fuzzing pipelines need deduplication, root-cause clustering, and triage discipline before output can be treated as security signal.
Editorial analysis by NHI Mgmt Group, based on content published by Appknox: “8.5 Billion Executions. 2 Real Bugs. Here’s Why.”.
By the numbers:
- The article says 357 crash files were reduced to 2 actual bugs.
Key questions
Q: What breaks when fuzzing crash counts are treated as vulnerabilities?
A: The workflow breaks because repeated crashes are not unique defects.
Q: Why do higher execution rates not guarantee better fuzzing results?
A: Higher execution rates increase the number of inputs tested, but they do not prove that the campaign reached new defect classes.
Q: What are the signs that a fuzzing campaign is producing duplicate noise?
A: A campaign is likely producing duplicate noise when coverage plateaus, crash counts keep rising and reproduction collapses many failures into a few stack traces.
Practitioner guidance
- Implement crash deduplication before triage Cluster AFL++ outputs by stack trace similarity and reproduction status so repeated failures do not inflate perceived risk.
- Use a mixed instrumentation matrix Run a native LTO binary for throughput, an ASAN build for validation and CmpLog for comparison-heavy paths instead of relying on one build.
- Separate coverage growth from defect validation Track whether new surfaces are reached, then verify whether each new crash is a distinct reproducible issue before escalating it.
Bottom line: Crash volume is a poor proxy for vulnerability discovery because one bug can manifest as many crash artefacts.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
18/08/2026 3:26 pm
Crash triage is the governance layer that turns fuzzing from telemetry into evidence. Security teams often invest in generation capacity first and interpretation second, but that sequence produces inflated confidence. If duplicate crashes are not clustered and reproduced, the programme is optimising noise, not assurance. The practitioner conclusion is simple: no crash should enter a risk workflow before it has been reduced to a unique, validated failure.
A few things that frame the scale:
- Only 5.7% of organisations have full visibility into their service accounts, according to the Ultimate Guide to NHIs.
A question worth separating out:
Q: How should teams judge whether fuzzing output is decision-grade?
A: Teams should ask whether each finding is reproducible, clusterable and traceable to a distinct root cause. If the answer is no, the output is still raw telemetry. Decision-grade fuzzing output is a small, stable set of unique defects that can be owned, fixed and retested.
👉 Read our full editorial: AFL++ crash triage shows why noise is not risk