Notifications
Clear all
Topic starter
06/09/2026 6:21 pm
TL;DR: Qwen 3.8 27B validated 45 findings at about $1.14 per finding and 6.85 million total tokens in MindFort’s NexBench, making it the most token-efficient model in the run, but its results skewed toward lower-severity issues and only 9 high-severity findings. That pattern matters because faster, cheaper models can expand continuous testing, yet they still need a harness and human governance to prove exploitability.
NHIMG editorial — based on content published by MindFort: How Good Is Qwen 3.8 For Cybersecurity?
By the numbers:
- Qwen 3.8 27B validated 45 findings in MindFort’s NexBench evaluation.
- Qwen 3.8 27B found those 45 findings on just 6.85 million total tokens.
- The model’s normalized cost worked out to about $1.14 per validated finding.
Questions worth separating out
Q: How should security teams use open-weight AI models for vulnerability testing?
A: Use them for large-scale reasoning, triage, and hypothesis generation, but keep exploit proof separate.
Q: Why does token efficiency not equal better security outcomes?
A: Because efficiency measures cost per output, not the depth or severity of the output.
Q: What do security teams get wrong about autonomous exploitation?
A: They often assume the problem is only faster detection.
Practitioner guidance
- Separate model capability from execution control Define the harness, browser access, container isolation, and scope manager as explicit security controls.
- Score findings by severity-weighted value Track low, medium, high, and critical outcomes separately, then prioritise models that produce reproducible high-severity results rather than raw volume.
- Require reproduction before triage Treat every AI-generated issue as a candidate until the finding is validated in a controlled environment.
What's in the full report
MindFort's full research covers the operational detail this post intentionally leaves for the source:
- Run-by-run NexBench scoring and how the validator reproduced each finding before acceptance
- The model-by-model comparison table showing where Qwen 3.8 27B sat against other systems on raw volume and severity
- The token and cost assumptions used to calculate findings-per-dollar and findings-per-token
- The evaluation setup for the isolated harness, including how the browser and container were scoped
👉 Read MindFort's evaluation of Qwen 3.8 27B for cybersecurity testing →
Qwen 3.8 27B for cybersecurity: where does cheap breadth stop?
Explore further
06/09/2026 6:21 pm
Cheap model inference is becoming a governance issue, not just a tooling choice. When a model can run repeatedly at low cost, the operational question shifts from whether it can find issues to whether the surrounding process can absorb, validate, and prioritise them. For security teams, the meaningful control is no longer model access alone but the governance around execution, scoping, and verification.
A question worth separating out:
Q: Should organisations prioritise low-cost AI testing before deeper automation?
A: Yes, if the goal is to increase testing cadence without expanding risk. Cheap models can support continuous discovery, but only when there is a mature process for reproduction, triage, and remediation. If those controls are absent, more automation simply creates more noise.
👉 Read our full editorial: Qwen 3.8 27B shows cheap breadth, not deep vulnerability chaining
