Topic starter
06/09/2026 9:34 am
TL;DR: Changing only the harness around the same model can create up to a 6x performance gap, showing that context management and orchestration matter more than raw model capability for security operations, according to Crogl. The real security issue is not model access but whether the surrounding control plane can preserve investigative context, avoid repetition, and keep decisions grounded in the right evidence.
Editorial analysis by NHI Mgmt Group, based on content published by Crogl: “Why Can’t I Just Vibe Code My Own Harness?”.
Key questions
Q: What breaks when an AI SOC harness cannot preserve investigation context?
A: The system starts repeating work, losing causal links, and overreacting to noisy tool output.
Q: Why do AI security workflows need explicit memory instead of relying on the model?
A: Because the model does not reliably retain case history across tool calls, and security work depends on knowing what has already been checked.
Q: How can teams tell whether an AI harness is actually improving analyst work?
A: Look for fewer repeated queries, cleaner evidence selection, and better continuity across investigation steps.
Practitioner guidance
- Define the investigation state model Map which entities, alerts, hosts, identities, and relationships the harness must persist across a case so repeated analysis is avoided.
- Limit raw context injection Replace large unfiltered tool outputs with staged retrieval and summarisation so the model sees only the evidence needed for the current reasoning step.
- Track previously analysed evidence Store case history outside the model and reference it explicitly so the harness can distinguish new signals from already reviewed artifacts.
Bottom line: AI SOC performance depends on the harness that controls context, memory, and evidence flow, not just the underlying model.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
06/09/2026 9:34 am
Harness scaffolding is now part of security control design, not a developer convenience. Once AI is used inside SOC workflows, the orchestration layer becomes the difference between repeatable investigation and unreliable inference. The article's central point is that the model is interchangeable, but the surrounding state management is not. Practitioners should treat harness design as a governance problem because it determines what evidence the system can trust.
A question worth separating out:
Q: How should security teams decide where to use AI first in the SOC?
A: Start with the layer that has the clearest operational pain and the cleanest success metric. Detection, triage, and response solve different problems, so the best first use case is usually the one where AI can reduce noise, improve analyst throughput, or speed containment without introducing opaque decision-making.
👉 Read our full editorial: AI harness scaffolding is the real gap in security operations