AI model selection for SecOps now hinges on reasoning efficiency

At a glance

What this is: Cotool’s benchmark update shows that two frontier models now reach the same 65% SecOps accuracy, but Opus 4.5 is much faster while GPT-5.1 is more cost efficient.

Why it matters: Security teams evaluating AI for SecOps should optimise for end-to-end investigation efficiency, because task speed, tool calls, and cost now influence operational viability as much as accuracy.

By the numbers:

• GPT-5.1 and Claude Opus 4.5 tied for the highest overall accuracy at 65% on the benchmark.
• Claude Opus 4.5 completed tasks in 122 seconds on average.
• GPT-5.1 averaged 14.5 tool calls per task.
• GPT-5.1 achieves top-tier accuracy at roughly $1.67 per task.

Context

SecOps model selection is no longer a simple accuracy contest. Once an AI agent is expected to investigate logs, correlate signals, and iterate through tools, the real governance problem becomes whether the model can complete the task efficiently enough to fit operational workflows.

For security teams, this is an identity-adjacent control issue as much as a model-evaluation issue. A SecOps agent that burns excessive tokens, spins through too many tool calls, or stalls on long-horizon tasks can create cost, reliability, and oversight problems even when headline accuracy looks strong.

Key questions

Q: How should security teams choose AI models for SecOps workflows?

A: Choose models by task, not by brand. Use higher-accuracy models for deep investigations, faster models for interactive triage, and lower-cost models for summarisation or routine enrichment. Then validate the choice with repeatable evals that measure accuracy, latency, tool use, and failure recovery under realistic telemetry conditions.

Q: Why can a slower model be better for security operations?

A: A slower model can still be better if it reaches the correct answer with fewer failed turns, less analyst intervention, and lower total investigation overhead. In SecOps, the relevant metric is time to useful decision, not just generation latency. Efficient reasoning often beats raw speed when the model must traverse long context and multiple tools.

Q: What are the signs that an AI model is inefficient in SecOps?

A: Look for high tool-call counts, repeated dead-end searches, frequent failure to complete tasks, and long tail latency on complex cases. A model can score well on accuracy while still creating operational drag if it needs too many steps to converge. Those signals usually mean the model is unsuitable for routine investigations.

Q: How should teams decide when to use cheaper models versus frontier models?

A: Use cheaper models for low-risk triage, enrichment, and summarisation when the cost of a miss is limited. Reserve frontier models for complex investigations, high-confidence conclusions, or response workflows where failed reasoning would slow containment. The decision should be based on task criticality, not on model brand or benchmark rank alone.

Technical breakdown

Why task completion and tool calls matter more than raw latency

In agentic SecOps, a model is not just answering a prompt. It is driving an investigation loop that may include search, log review, hypothesis testing, and follow-up actions through tools. Raw inference latency captures only one slice of that workflow. A model that reasons efficiently can produce fewer dead-end turns, fewer tool calls, and a more stable path to resolution, even if its token generation is not the fastest. That is why task completion rate and tool efficiency are better indicators of operational usefulness than speed alone. In long-horizon security work, the quality of the reasoning path often determines the total cost of the investigation.

Practical implication: Measure SecOps AI by completed investigations per unit time, not just single-response latency.

How accuracy and cost form a practical Pareto frontier

The benchmark results show why model choice in SecOps is becoming a resource-allocation decision. Accuracy gains matter, but once two models are close on performance, the cost per task and the number of interactions required to finish the job become the deciding variables. That is the practical meaning of a Pareto frontier in this context: no single model dominates every dimension, so teams have to choose the least-bad trade-off for the use case. For alert triage, low-cost and fast models may suffice. For deeper investigations, higher-cost models can still be justified if they reduce analyst effort and close cases more reliably.

Practical implication: Create separate model tiers for triage, investigation, and high-consequence response tasks.

Why long-context SecOps workflows expose model weaknesses

Security investigations often involve long context windows, multiple data sources, and repeated tool use. That combination exposes failures that simple benchmarks miss, especially when models must retain evidence across many turns without drifting or over-searching. A model can look strong on accuracy while still underperforming operationally if it needs too many retries or loses coherence mid-task. This is where agent design and model design meet: the model must support the orchestration pattern, not just the benchmark score. The article’s results suggest that some models are better at converging quickly, while others are better at throughput or steady-state consistency.

Practical implication: Test models against multi-step incident workflows before allowing them into production investigations.

NHI Mgmt Group analysis

Reasoning efficiency is becoming a governance variable, not just a model-quality metric. In SecOps, the key question is no longer whether a model can answer correctly in isolation. It is whether it can do so with enough efficiency, stability, and observability to fit an operational control environment. Practitioners should treat investigation throughput as part of the control design, not an implementation detail.

SecOps model selection now needs workload-specific segmentation. The benchmark reinforces that one model will not be optimal across triage, enrichment, and deep investigation. That pushes teams toward policy-based model routing, where lower-cost models handle simple work and stronger reasoning models handle high-friction cases. The governance challenge is deciding where those boundaries sit and how exceptions are approved.

Model efficiency should be judged by tool behaviour, not vendor narratives. A model that completes tasks in fewer turns can reduce analyst burden even if its raw generation speed is not the lowest. This is a subtle but important distinction for AI governance: the unit of control is the task, not the token. Practitioners should insist on workflow-level metrics before scaling agentic SecOps.

For SecOps, the new failure mode is inefficient competence. A model can be accurate enough to look production-ready while still being too slow, too expensive, or too erratic for real operations. That creates a procurement trap where teams buy capability that does not translate into usable incident response capacity. The right question is whether the model reduces time to decision under realistic case load.

AI-driven SecOps should be governed like an operational dependency, not an experiment. Once models are used to investigate alerts and guide response, their latency, reliability, and cost profile become part of resilience planning. Teams need thresholds for acceptable completion rate and cost per task before models are allowed into routine workflows.

From our research library:

• 53% of security leaders expect AI to run major portions of their infrastructure autonomously within the next three years, according to the 2026 Infrastructure Identity Survey.
• 19% of organisations give AI systems dramatically more access than human employees, nearly one in five granting unrestricted privilege, according to the 2026 Infrastructure Identity Survey.
• Read next: Agentic AI Security Guide

What this signals

Task-level efficiency is now the meaningful control plane for SecOps AI. Teams should expect model selection to shift from one-off benchmark comparisons toward workflow governance, where completion rate, tool usage, and time-to-decision define operational fit. The practical test is whether the model can support analysts without inflating cost or slowing containment.

Reasoning efficiency changes how model risk should be segmented. A model that converges quickly on investigation tasks can be operationally superior even if another model is marginally faster on a single response. That means SecOps programmes need routing policies, case complexity thresholds, and ongoing evaluation against live workflows rather than static scorecards.

The 53% of security leaders who expect AI to run major portions of their infrastructure autonomously within three years should read this benchmark as a warning: autonomy without efficiency and completion discipline can become expensive very quickly, according to the 2026 Infrastructure Identity Survey.

For practitioners

• Define workload tiers for SecOps models Assign separate models to alert triage, enrichment, and deep investigation so that expensive reasoning is reserved for cases that justify it.
• Benchmark end-to-end investigation flows Test models against full SecOps tasks, including tool use, multi-step reasoning, and case completion, rather than relying on single-turn benchmarks.
• Set cost and completion thresholds Establish acceptable limits for cost per task, task completion rate, and average investigation time before moving a model into production workflows.
• Monitor tool-call inflation Track how many tool invocations each model needs to solve the same class of incident, because excessive calls often signal inefficient reasoning.
• Route high-consequence cases to stronger models Use the most reliable model for time-critical investigations where decision quality matters more than marginal cost savings.

Key takeaways

• The benchmark shows that SecOps model choice should be governed by end-to-end investigation efficiency, not by latency alone.
• GPT-5.1 and Claude Opus 4.5 reached the same 65% accuracy, but they achieved it through different operational trade-offs that matter to security teams.
• Teams adopting AI for SecOps need workload-specific routing, cost thresholds, and workflow testing before they treat model output as production-ready.

Key terms

• SecOps Agent: A SecOps agent is an AI system that uses security tools to investigate alerts, query telemetry, and assist with operational tasks. Its value depends on repeatable performance, bounded access, and auditable behaviour, not just model capability.
• Task Completion Rate: The percentage of benchmark tasks a model finishes successfully within the evaluation constraints. For security automation, completion rate is a critical indicator because partial investigations can create more manual work, delay incident handling, and produce misleading confidence in the model’s output.
• Tool-call Efficiency: Tool-call efficiency measures how much work an AI system can accomplish per external action it takes. Lower call counts can indicate better reasoning, but only when the model still reaches the right outcome, so this metric should be read alongside completion rate and accuracy.
• Pareto Frontier: The set of options where improving one outcome, such as speed or cost, would worsen another, such as accuracy. In AI operations, this concept helps teams choose a model that fits the use case rather than chasing a single best score across all dimensions.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and agentic AI identity for practitioners who need to govern dynamic access patterns. It helps security and identity teams connect operational controls to the broader risks of delegated and autonomous systems.