TL;DR: Frontier-model-assisted vulnerability hunting in OT security software can surface hundreds of candidates, according to Dragos, but 60 to 90 percent of raw findings die before human review because most lack reachability, valid trust boundaries, or proof. The real problem is not discovery volume but verification discipline: without a threat model and refutation gates, signal quality collapses and defenders trust noise.
At a glance
What this is: Dragos describes a methodology for using AI to hunt vulnerabilities in OT security software while forcing every candidate through adversarial verification.
Why it matters: It matters because OT and industrial security teams need to separate genuine exploit paths from AI-generated noise before they trust automated vulnerability discovery in critical infrastructure tooling.
Context
AI-assisted vulnerability discovery is only useful when the results are trustworthy, because large language models can generate plausible but wrong findings faster than engineers can inspect them. In industrial environments, that problem matters more because the software under test often protects systems tied to power, water, and gas, so false confidence in a finding pipeline can distort remediation priorities and waste scarce engineering time.
The core governance gap is verification, not generation. Dragos frames the work as a threat hunting guide problem translated into secure software analysis: build a threat model, test reachability, refute weak claims, and prove only what survives repeated challenge. That is a stronger model for OT security software than treating AI output as triage-ready evidence.
Key questions
Q: What breaks when AI detection has no current threat intelligence?
A: The main failure is false confidence. Teams may score anomalies accurately yet still not know whether the signal represents routine noise, a known campaign, or a threat aimed at their organisation. Without current intelligence, responders spend time on the wrong events and miss the ones most likely to progress into intrusion or abuse.
Q: Why do AI-generated vulnerability reports create more triage friction than value?
A: They create friction because models can produce fluent attack narratives without confirming target state, exploitability, or version fit. Triage teams must then spend time reproducing issues that may be hallucinated, which diverts effort from real vulnerabilities. Over time, the programme starts paying a tax on polish instead of rewarding evidence.
Q: What are the signs that an AI remediation pipeline is failing?
A: Look for repeated PRs on low-value findings, conflicting severity signals, manual triage after the agent has already acted, and unresolved issues that never make it into the pipeline. Those signs show the agent is consuming noisy telemetry rather than improving security decisions. A healthy workflow should reduce review burden, not add another queue for engineers to manage.
Q: What should OT security teams do before trusting AI-assisted vulnerability discovery?
A: They should validate the workflow against known cases and confirm that the test fixture actually reaches the code under review. If the setup cannot exercise the target path, every later result is suspect, no matter how expensive the run looked. In OT environments, that calibration step is essential because access gates and environment mismatch can invalidate an entire campaign.
Technical breakdown
Why AI vulnerability hunters overproduce false positives
A frontier model can identify many candidate flaws in a codebase, but most are not exploitable. The common failure modes are simple: the sink is protected by a control layer the model missed, the path is dead, or the attack assumes privileges that do not exist. In OT security software, where trust boundaries and exposed interfaces matter more than raw code patterns, a candidate is only meaningful if an attacker can actually reach it and use it to cross a boundary. That is why candidate generation and exploitability are different tasks, and why a high raw hit rate tells you almost nothing about practical risk. See the OT Security Guide.
Practical implication: Treat AI-generated vulnerability lists as hypotheses that still need reachability and exploitability checks.
How threat models turn AI findings into usable evidence
Dragos builds the threat model first and uses it as the map for both hunting and refutation. The model captures trust boundaries, entry points, privileged operations, data flows, and mitigations already present, so a verifier does not have to reconstruct the system from scratch each time. That matters because the quality of a finding depends on context: the same code path can be trivial, unreachable, or blocked by a control already in place. A good threat model therefore becomes the shared reference point that lets multiple agents evaluate the same claim consistently instead of drifting toward different answers. Read Threat Modelling AI Agents.
Practical implication: Create a structured threat model before large-scale AI-assisted reviews so verifiers can judge claims against the same system context.
Why proof and judgment must sit after detection
Detection alone is too weak for security software analysis. Dragos puts survivors through proof, where an exploit-dev agent must demonstrate a repeatable observable effect in an isolated environment, and then judgment, where another agent attacks the argument itself. This sequencing matters because many claims sound plausible until they are forced to reproduce from clean state or survive criticism about preconditions, lab-only artifacts, or misread severity. In practice, this creates a filter that reduces severity theatre and makes the final report defensible to engineers who will otherwise dismiss unverified claims.
Practical implication: Require repeatable proof and a separate skepticism stage before you assign severity to AI-found vulnerabilities.
Threat narrative
Attacker objective: The objective is to find exploitable weaknesses in OT security software that could weaken the defenses protecting critical infrastructure.
- Entry begins with AI-generated candidate discovery against OT security software code and configuration surfaces, including exposed services and trust boundaries.
- Escalation occurs when the team tests whether a candidate can move from theoretical flaw to a reachable path that crosses a privilege or trust boundary.
- Impact is reached only when a candidate survives refutation and reproduces in a pinned environment as a repeatable, observable exploit chain.
Breaches seen in the wild
- CI/CD pipeline exploitation case study: Credentials in an exposed .git/config let a researcher edit a Bitbucket pipeline so it planted their SSH key on the server. No victim was named.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
Signal quality is the real security control here, not candidate volume. AI can flood a team with plausible findings, but trust in the result depends on how aggressively the pipeline kills weak claims. In OT security software, that matters because the downstream cost of a false positive is not just wasted engineering time, but distorted risk prioritisation around systems that support critical infrastructure. The practitioner lesson is to measure verification quality, not discovery throughput.
Threat modelling is the governance layer that makes AI-assisted analysis repeatable. Without a shared model of trust boundaries, exposed entry points, and existing mitigations, every verifier will rediscover the system differently. That creates inconsistent outcomes and turns the validation layer into a lottery. The stronger practice is to treat the threat model as the canonical map for industrial identity and access and software trust boundaries alike, then force every finding to survive the same context.
Proof-of-exploit discipline is what separates analysis from security theatre. A finding that cannot reproduce from clean state is not ready for engineering action, regardless of how persuasive the reasoning sounds. This is especially important in critical infrastructure software, where teams cannot afford to spend cycles on claims that collapse under a real environment. The practitioner conclusion is straightforward: if it cannot be shown, it is not yet a vulnerability.
AI changes the asymmetry only when defenders use it with better fuel than attackers. The advantage comes from source access, design context, live lab control, and a threat-model library, not from the model itself. That means the enduring governance question is whether your team can supply enough system context to make the AI useful without letting it manufacture certainty. The right posture is controlled augmentation, not blind automation.
OT security teams should treat AI-assisted review as a new assurance workflow, not a replacement for review. The methodology here points toward a future where code review, fuzzing, threat modelling, and exploit validation are chained together as one disciplined system. That will matter most for organisations protecting industrial systems, where the cost of both false negatives and false positives is high. The practitioner implication is to build an evidence pipeline, not a prompt pipeline.
What this signals
Verification debt is the hidden risk in AI-assisted security tooling. Teams can scale candidate generation quickly, but every additional candidate also increases the burden on the refutation layer. When the verifier is weak, the organisation mistakes volume for coverage and misses the governance problem entirely: who decides what is real enough to act on.
OT security analysis is moving toward evidence pipelines, not point-in-time reviews. The methodology here suggests that model quality, threat-model quality, and lab fidelity will matter more than raw automation count. For practitioners, the signal is clear: if your pipeline cannot reproduce, refute, and re-run, it is not ready for critical infrastructure software.
For practitioners
- Build a threat model before large AI-assisted hunts Map trust boundaries, entry points, privileged operations, and existing mitigations before you let agents generate candidates, so verifiers evaluate claims against the same system context. Read Threat Modelling AI Agents.
- Separate recall from verification Use one stage to generate broad candidate coverage and a different stage to refute claims with a bare hypothesis, so persuasive reasoning does not bias the skeptic.
- Require reproducible proof in an isolated environment Only accept findings that reproduce from clean state against pinned builds or replicas, because plausible prose is not evidence of exploitability.
- Calibrate on known results before scaling Test the workflow against previously analysed high-severity cases and fix any ground-truth errors before widening the search to new repositories or services.
- Track the false-positive death rate as a quality metric Measure how many raw candidates fail at each kill point so you can see whether signal quality is improving or the pipeline is drifting toward noise.
Key takeaways
- AI-assisted vulnerability discovery in OT security software creates a trust problem as much as a detection problem.
- Dragos says most raw candidates never survive refutation, proof, and judgment, which makes verification the core control.
- Teams that want usable results need threat models, reproducible proof, and calibration against known cases before they scale automation.
Related NHI Mgmt Group resources: OT Security Guide · Threat Modelling AI Agents · Threat Hunting Guide
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK addresses the attack and risk surface, while NIST CSF 2.0, CIS Controls v8, NIST SP 800-53 Rev 5 and OWASP SAMM set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| MITRE ATT&CK | TA0006; TA0008 — Credential Access; Lateral Movement | The article frames attack progression through reconnaissance, escalation, and movement across trust boundaries. |
| Recommendation — Map AI-found weaknesses to ATT&CK tactics and validate whether they enable reachability, escalation, or lateral movement. | ||
| NIST CSF 2.0 | ID.RA-01 — Identified Assets, Risks, and Vulnerabilities are Inventoryed | Threat modelling and candidate validation depend on inventorying trust boundaries and attack surfaces. |
| Recommendation — Inventory exposed trust boundaries and high-value paths before you let AI hunters evaluate code. | ||
| CIS Controls v8 | CIS-16 — Application Software Security | The topic is secure software review and validation rather than governance in the abstract. |
| Recommendation — Apply secure software review practices that force findings to survive proof, not just static reasoning. | ||
| NIST SP 800-53 Rev 5 | SA-11 — Developer Testing and Evaluation | The workflow is a testing and verification regime for code under review. |
| Recommendation — Use SA-11 to require repeatable verification of security findings before remediation begins. | ||
| OWASP SAMM | Security Testing — Security Testing | The article is about building a repeatable security testing practice around AI-assisted review. |
| Recommendation — Embed AI-assisted vulnerability review into a mature security testing practice with calibration and proof gates. | ||
Key terms
- Threat Modeling: A structured way to identify where an application can be entered, abused, or misused before or during development. It maps trust boundaries, access points, and likely abuse cases so teams can design controls that match the way attackers and misuse actually happen.
- Refutation Gate: A refutation gate is a control point where a claimed weakness is actively challenged before it can proceed. The purpose is to kill weak or misleading findings early, so downstream proof and engineering time are reserved for issues that still hold up under scrutiny.
- Proof-of-exploit: Evidence that a suspected weakness can actually be used to achieve the expected security outcome. In practice this includes the request, response, and reproduction details needed to confirm the issue independently, which separates actionable findings from unverified hypotheses.
- Validation Pipeline: A structured process that proves whether a model-generated finding is truly exploitable before it is reported. It tests the live target, captures request and response evidence, escalates ambiguous results, and discards failures. This prevents confident but wrong outputs from becoming false positives in production reporting.
Deepen your knowledge
NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management for practitioners who need a stronger control baseline. It is relevant when identity, privilege, and credential discipline shape the security programme that underpins wider detection and verification work.
Published by the NHIMG editorial team on October 22, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org