TL;DR: AFL++ campaigns can generate billions of executions and thousands of crashes while still yielding only a handful of real bugs, according to Appknox, because crash volume, coverage, and execution speed do not equal vulnerability discovery. The lesson is that fuzzing pipelines need deduplication, root-cause clustering, and triage discipline before output can be treated as security signal.
At a glance
What this is: Appknox argues that AFL++ fuzzing output becomes meaningful only after crash triage, because massive crash counts can collapse to a very small number of unique bugs.
Why it matters: For security and IAM practitioners, the lesson is that tool output must be operationalised into deduplicated, root-caused findings before it can inform risk decisions or engineering priority.
By the numbers:
- The article says 357 crash files were reduced to 2 actual bugs.
Context
Fuzzing is a control discovery technique, not a vulnerability verdict. When coverage-guided testing produces large crash volumes, teams still have to separate duplicate failures, reachable parser paths and reproducible defects before the results can inform engineering priority.
The article’s core governance problem is interpretation: output volume can look like security progress even when it mostly reflects repeated execution of the same failure mode. For application security teams, and for any programme that uses automation to surface risk, the real question is whether findings are being converted into validated, decision-grade evidence.
Although the article is about mobile and application testing rather than identity, the governance lesson carries into NHI and agentic AI programmes as well. Automation only creates value when outputs are deduplicated, classified and assigned to the right owner for action.
Key questions
Q: What breaks when fuzzing crash counts are treated as vulnerabilities?
A: The workflow breaks because repeated crashes are not unique defects. A single parsing bug can generate hundreds of artefacts across different inputs and workers, which makes the campaign look more productive than it is. Security teams need reproduction and clustering before they can say whether the output reflects real exposure or duplicated noise.
Q: Why do higher execution rates not guarantee better fuzzing results?
A: Higher execution rates increase the number of inputs tested, but they do not prove that the campaign reached new defect classes. If the harness, mutation strategy and coverage surface stay narrow, the fuzzer can simply find the same fault faster. Real improvement comes from deeper surfaces plus better triage, not speed alone.
Q: What are the signs that a fuzzing campaign is producing duplicate noise?
A: A campaign is likely producing duplicate noise when coverage plateaus, crash counts keep rising and reproduction collapses many failures into a few stack traces. That pattern suggests the harness is still useful, but the pipeline is no longer discovering proportionate new risk. The signal is consolidation, not volume.
Q: How should teams judge whether fuzzing output is decision-grade?
A: Teams should ask whether each finding is reproducible, clusterable and traceable to a distinct root cause. If the answer is no, the output is still raw telemetry. Decision-grade fuzzing output is a small, stable set of unique defects that can be owned, fixed and retested.
Technical breakdown
Why AFL++ crash counts overstate discovery
AFL++ is coverage-guided fuzzing, which means it mutates inputs based on observed execution paths. A single underlying defect can be reached through many different mutations, so one bug often produces hundreds of crash artefacts. Parallel instances amplify that duplication further. Crash counts therefore measure how often a fault was triggered, not how many distinct weaknesses exist. Without stack-trace clustering, reproduction and root-cause grouping, fuzzing output is a noisy proxy for security rather than a reliable indicator of it.
Practical implication: Treat crash counts as raw telemetry until deduplication and reproduction turn them into unique defects.
Why throughput and coverage are not the same control
The article separates execution speed from meaningful discovery. Higher exec/sec can increase the number of mutated inputs seen by the target, but that does not automatically expand the quality of coverage or the novelty of bugs found. Persistent mode, LTO, CmpLog and mixed power schedules change the efficiency of the search, yet they still need an analysis layer to determine whether new execution regions were actually reached. In other words, throughput is an input to discovery, not proof of it.
Practical implication: Measure whether new execution regions are reached, not just whether the fuzzer is running faster.
Why crash triage is the security control that matters
The campaign’s value emerged only after structured triage: reproduce crashes with ASAN, deduplicate outputs, generate CASR reports and cluster by stack trace similarity. That process converts hundreds or thousands of noisy failures into a small set of actionable defects. This is the same governance pattern many security programmes miss elsewhere. Detection is not the end state. Validation, classification and assignment determine whether a tool produces risk intelligence or just an oversized queue.
Practical implication: Build triage into the workflow so automated findings become owned, reproducible issues rather than undifferentiated noise.
Threat narrative
Attacker objective: The objective is to trigger parser failures that look like many distinct issues even when they collapse to a small number of root causes.
- Entry occurs when malformed archive inputs are fed into attacker-controlled parsing paths in libarchive.
- Credential or privilege abuse is not the issue here; instead, repeated mutations trigger the same parser faults through multiple input paths.
- Escalation appears as duplicate crash generation, which creates the illusion of widespread vulnerability when the underlying defect count remains low.
- Impact is decision failure, because untriaged crash volume can misdirect engineering effort and distort real risk prioritisation.
Breaches seen in the wild
- iOS apps leaking hard-coded secrets: Cybernews found 71% of 156,080 iOS apps leak hard-coded secrets, with open cloud storage and Firebase databases exposing user data.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
Crash triage is the governance layer that turns fuzzing from telemetry into evidence. Security teams often invest in generation capacity first and interpretation second, but that sequence produces inflated confidence. If duplicate crashes are not clustered and reproduced, the programme is optimising noise, not assurance. The practitioner conclusion is simple: no crash should enter a risk workflow before it has been reduced to a unique, validated failure.
Execution speed is a search efficiency metric, not a security outcome. The article shows that billions of executions and thousands of crashes can still map to only two real bugs. That matters because many programmes still equate more output with more coverage of risk. In practice, coverage depth, surface selection and triage quality matter more than raw volume. The practitioner conclusion is to manage fuzzing as a measurement system, not as a scoreboard.
Execution-aware security needs root-cause discipline, not finding accumulation. The named concept here is noise-to-signal collapse: the point at which a large automated finding set is reduced into a small actionable set by deduplication and classification. That pattern applies directly to AppSec, but also to any automated security workflow that can overproduce alerts. The practitioner conclusion is to design the workflow around decision quality, not output size.
Crash inflation is a control failure when teams mistake repeated triggers for distinct weaknesses. Appknox’s example shows that without clustering, one defect can masquerade as many. That creates false urgency, skews prioritisation and can hide the true root cause behind repeated symptom reporting. The practitioner conclusion is to define unique-failure criteria before the campaign starts.
Fuzzing programmes should be judged by validated defects, not by raw artefact counts. The article’s own numbers show why governance should focus on the final triage funnel rather than the initial crash storm. When the end state is a few reproducible bugs with clear stack traces, the pipeline is doing useful work; when the end state is a queue of duplicates, it is not. The practitioner conclusion is to make unique defect yield the programme KPI.
From our research library:
- Only 5.7% of organisations have full visibility into their service accounts, according to the Ultimate Guide to NHIs.
- Read next: Top 10 NHI Issues
What this signals
Noise-to-signal collapse: fuzzing teams should expect the first pass to overproduce artefacts and should design the workflow so triage, not generation, determines what reaches remediation. That means pairing harness quality with deduplication and root-cause review, otherwise the programme will mistake repeated crashes for expanding exposure.
Automation-heavy security programmes fail when they optimise output volume instead of validated outcomes. For AppSec teams, the practical shift is to measure unique defect yield, not raw crash counts; for identity and NHI teams, the same principle applies to alerts, discoveries and inventory feeds that only become useful after ownership and classification.
For practitioners
- Implement crash deduplication before triage Cluster AFL++ outputs by stack trace similarity and reproduction status so repeated failures do not inflate perceived risk.
- Use a mixed instrumentation matrix Run a native LTO binary for throughput, an ASAN build for validation and CmpLog for comparison-heavy paths instead of relying on one build.
- Separate coverage growth from defect validation Track whether new surfaces are reached, then verify whether each new crash is a distinct reproducible issue before escalating it.
- Tune parallel fuzzers to avoid redundant search Use different power schedules across instances so secondary fuzzers explore complementary regions rather than converging on the same mutations.
- Define unique-bug criteria up front Require reproduction, stack clustering and root-cause confirmation before any crash is counted as a security issue.
Key takeaways
- Crash volume is a poor proxy for vulnerability discovery because one bug can manifest as many crash artefacts.
- The article’s campaign shows why deduplication and stack-based clustering are necessary before fuzzing output can guide remediation.
- The control that changes the result is triage discipline, not more raw executions.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK addresses the attack and risk surface, while CIS Controls v8 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| MITRE ATT&CK | TA0007 — Discovery | The article is about surfacing and validating defects through execution analysis. |
| Recommendation — Map fuzzing coverage to discovery objectives and confirm whether new paths actually expose distinct defects. | ||
| CIS Controls v8 | CIS-16 — Application Software Security | The article focuses on application testing and defect validation in the SDLC. |
| Recommendation — Use application security testing controls to require reproduction, triage and ownership for each unique finding. | ||
| NIST CSF 2.0 | PR.IP-12 — Vulnerability Management Plan | The workflow depends on turning raw findings into managed, validated remediation work. |
| Recommendation — Integrate fuzzing triage into vulnerability management so only validated defects enter remediation queues. | ||
Key terms
- Coverage-guided fuzzing: A testing method that mutates inputs based on how the target program executes. The goal is to explore new code paths and expose failure states, but the raw output still needs triage before it can be treated as a security finding.
- Crash Deduplication: Crash deduplication is the process of grouping repeated failures that stem from the same underlying bug. In fuzzing, it prevents one defect from being counted many times and helps teams focus on unique root causes that actually change security posture.
- Persistent Mode: Persistent mode is a fuzzing execution pattern where the target stays alive across many test iterations instead of restarting each time. It improves throughput, but it only produces reliable results when the harness correctly resets state between loops.
- Stack trace clustering: A method for grouping crashes by similar call stacks so teams can identify whether failures represent the same bug or multiple distinct issues. It is one of the most practical ways to convert noisy fuzzing output into actionable engineering work.
Deepen your knowledge
NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It is suitable for practitioners who need to connect governance discipline to identity risk across human and non-human programmes.
Published by the NHIMG editorial team on August 18, 2026.
Updated on October 12, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org