Why does AI need clear baselines in a security operations program?

Why This Matters for Security Teams

AI in a security operations program is only useful if leaders can tell whether it is improving detection, triage, and response or simply adding a new layer of noise. Baselines turn that judgement into evidence. Without them, teams often praise automation for speed while missing hidden costs such as analyst rework, false confidence, or longer handoffs between tools and people. That makes governance harder and weakens trust in the operating model.

For security operations, the baseline should reflect the actual work being done before AI changes the process. That includes alert volume, mean time to acknowledge, investigation effort, escalation rates, and the amount of time spent suppressing duplicate or low-value alerts. NIST Cybersecurity Framework 2.0 is useful here because it encourages a structured view of current capability before pursuing improvements, rather than treating transformation as a purely technical exercise. The point is not to measure everything, but to measure the few indicators that expose whether the SOC is becoming more resilient.

In practice, many security teams only discover that AI has shifted workload after analysts are already compensating for it informally.

How It Works in Practice

A workable baseline is built from the security operations process as it exists today, not from an ideal future-state design. That means collecting a consistent pre-deployment sample over enough time to include normal variation, peak periods, and at least a few representative incident types. If the program uses SOAR, SIEM, or case management tooling, the baseline should separate raw machine activity from human handling so the team can see where effort is actually spent.

Useful baseline measures usually fall into three groups:

• Response timing, such as time to acknowledge, time to investigate, and time to contain.
• Workload indicators, such as alert volume, queue depth, analyst touch count, and rework caused by false positives.
• Quality indicators, such as missed escalations, repeat incidents, and inconsistency in disposition decisions.

Once AI is introduced, the same measures should be reviewed on the same cadence. The comparison matters more than the absolute number. If alert closure is faster but escalations increase later, the baseline has exposed a tradeoff rather than a simple win. That is why current guidance suggests pairing operational metrics with governance checks for model drift, decision traceability, and analyst override rates. Those controls make it possible to tell whether AI is supporting the SOC or quietly becoming another unmonitored dependency.

A practical discipline is to define which parts of the workflow AI may influence, which decisions remain human-owned, and what evidence must be retained when the system changes a recommendation. Where this is done well, the baseline becomes the reference point for both performance and accountability. These controls tend to break down when log quality is poor or incident handling is highly manual because the program cannot separate AI impact from pre-existing process variability.

Common Variations and Edge Cases

Tighter baseline measurement often increases analyst overhead, requiring organisations to balance visibility against the time spent collecting and validating metrics. That tradeoff is acceptable when AI is moving into decision support, but it can feel heavy in small SOCs that already operate near capacity.

Best practice is evolving for environments where AI is only used in narrow tasks, such as phishing triage or enrichment. In those cases, a full enterprise baseline may be unnecessary, but a task-specific baseline is still essential. For example, a team may only need to track the number of items enriched, time saved per case, and the rate of analyst correction. The same applies where multiple tools share responsibility: if the SIEM, SOAR, and AI layer all influence the outcome, the baseline must be scoped tightly enough to avoid attributing improvements to the wrong control.

There is also a governance edge case when AI output informs executive reporting but does not directly change alert handling. Even then, baseline drift matters because leadership decisions may be based on metrics that the AI helped compress or reframe. In those scenarios, the safest approach is to keep a pre-AI reporting view alongside the AI-assisted view so trend lines remain comparable over time.

Related resources from NHI Mgmt Group

• What is the difference between advisory AI and agentic AI in security operations?
• What breaks when organisations treat AI governance as a separate security program?
• How can security teams reduce risk as AI becomes more common in IT operations?
• What should teams do when AI tools are used in security operations?