Topic starter
15/08/2026 12:47 pm
TL;DR: Agentic SOCs can produce confident, polished triage narratives while still missing the evidence needed for a correct disposition, according to D3’s account of a BSides Pittsburgh talk by John Dwyer. The decisive control is not model confidence but whether investigations are evidence-led, replayable, and grounded in deterministic classification before the LLM summarizes anything.
Editorial analysis by NHI Mgmt Group, based on content published by D3: “Anyone Can Build An Agentic SOC, But Will It Be Accurate?”.
Key questions
Q: What breaks when an agentic SOC summary is generated before evidence is collected?
A: The investigation can become persuasive without being provable.
Q: Why do deterministic alert classifications matter in agentic SOC workflows?
A: They establish the evidence collection path before the model starts reasoning.
Q: How should security teams judge whether an agentic SOC investigation is trustworthy?
A: Teams should judge trustworthiness by whether the investigation exposes evidence, context, confidence, and missing data in a form a reviewer can challenge.
Practitioner guidance
- Define deterministic alert genus rules Classify common alerts such as process execution and OAuth identity events before any retrieval or summarisation begins.
- Require an evidence graph before triage output Block narrative generation until the system has assembled the core artefacts, including related event chains, account context, and supporting telemetry.
- Test replayability with identical payloads Run the same alert through the workflow multiple times and compare the verdict, evidence set, and escalation path.
Bottom line: Agentic SOCs can look authoritative while still producing unsupported verdicts, which makes evidence capture the real control objective.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
15/08/2026 12:47 pm
Evidence-first investigation is now a governance requirement, not a design preference. The core risk in agentic SOCs is not that the model fails to write a good summary. It is that the summary becomes the surrogate for proof, and teams mistake fluent output for validated investigation. That changes the control objective from message quality to evidence sufficiency, which is a materially different security problem. Practitioners should evaluate whether the workflow can prove disposition before it explains it.
A question worth separating out:
Q: Should SOC teams let context shape a case before the evidence graph is built?
A: No. Context should refine a case, not define it. If prior tickets, documentation, or historical false positives are retrieved first, the system can inherit stale assumptions and misread a current alert. The evidence graph must come first so context is weighted against facts, not above them.
👉 Read our full editorial: Agentic SOCs need evidence-first rails, not confident summaries