Topic starter
06/09/2026 6:21 pm
TL;DR: Qwen 3.8 27B validated 45 findings at about $1.14 per finding and 6.85 million total tokens in MindFort’s NexBench, making it the most token-efficient model in the run, but its results skewed toward lower-severity issues and only 9 high-severity findings. That pattern matters because faster, cheaper models can expand continuous testing, yet they still need a harness and human governance to prove exploitability.
Editorial analysis by NHI Mgmt Group, based on content published by MindFort: “How Good Is Qwen 3.8 For Cybersecurity?”.
Key questions
Q: How should security teams evaluate AI-assisted pentesting beyond raw finding counts?
A: Measure whether the system can preserve state, maintain authentication context, and reproduce a full attack path.
Q: Why do cheap security models often miss high-severity exploit chains?
A: High-severity findings usually require sustained context across multiple steps, including authentication, pivoting, and confirmation against a live target.
Q: What breaks when an AI testing harness is too thin?
A: The model may still identify suspicious code or likely weaknesses, but it cannot reliably execute, scope, and verify a complete exploit sequence.
Practitioner guidance
- Define exploit-depth acceptance criteria Require every AI-assisted security test to demonstrate preserved context across authentication, enumeration, and exploitation before the result is counted as actionable.
- Separate breadth from severity in reporting Track raw findings, validated findings, and high-severity findings as distinct metrics so cheap volume does not mask weak chaining performance.
- Instrument the harness as a governed control Log browser, container, and manager actions so reviewers can reconstruct how the model reached each validated finding.
Bottom line: Cheap breadth can be useful for security testing, but it does not prove that a model can carry an exploit chain far enough to matter.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
06/09/2026 6:21 pm
Cheap AI testing creates governance debt when teams confuse finding volume with exploit assurance. A model that produces many accepted findings can still underperform where it matters most, because chained exploitation depends on state retention, timing, and authenticated context. The field should stop treating raw issue counts as a proxy for offensive realism and instead measure whether a system can prove compromise paths end to end. Practitioners should align model evaluation with evidentiary quality, not token economy alone.
A question worth separating out:
Q: How should organisations decide whether to trust AI-generated vulnerability findings?
A: Trust them only when the workflow shows controlled execution, reproducibility, and validation of severity. Findings should be reviewed in the context of the test harness, the target state, and the evidence that the issue survives real conditions. If those elements are missing, the output should be treated as investigative noise, not assurance.
👉 Read our full editorial: Qwen 3.8 27B shows cheap breadth, not deep vulnerability chaining