AI identity attacks are testing Windows control assumptions

At a glance

What this is: This is Cotool’s August 2026 benchmark on simulated Windows identity-tier attacks, and its key finding is that identity telemetry is informative but still difficult to interpret reliably under benign lookalikes.

Why it matters: It matters to IAM and security teams because password spray, persistence, and service-credential abuse all sit at the boundary between identity governance and detection engineering, where false assumptions about normal directory activity create blind spots.

By the numbers:

• The password-spray hunt averaged 33.5% across the field, making it the hardest task in the benchmark.
• 4.6 led the benchmark at 77.6% accuracy, uracy, six points ahead of Claude Opus 4.8 at 71.5%.
• Four simulated Windows identity-tier attacks were scored across 14 models in 168 runs.

👉 Read Cotool's full analysis of BlueBench-Simulation-002 identity-tier attacks

Context

Windows identity-tier security depends on separating malicious account activity from the routine changes that happen in every enterprise directory. That is difficult because password resets, bulk administration, approved host changes, and service authentication can all resemble intrusion signals at first pass. In this benchmark, the challenge is not only detection quality but the ability to reason about identity telemetry without overcalling normal operations.

For IAM and directory teams, the article is really about the governance gap between what identity controls permit and what telemetry can prove under pressure. The benchmark sits squarely in the overlap between human identity administration, privileged access, and non-human identity exposure through service credentials, which is where many enterprise detection programmes are weakest.

Key questions

Q: Why do password-spray attacks often look like normal directory activity at first?

A: Password spray can resemble routine identity noise because lockouts, password rotations, and approved bulk operations all create bursts of authentication activity. The practical test is not whether volume is high, but whether the pattern aligns with an authorised lifecycle event and an expected operational purpose. Without that context, teams risk escalating benign administration as an intrusion.

Q: What breaks when service identities are not tracked as a distinct control group?

A: Detection logic loses the ability to separate normal service authentication from credential theft. Service accounts often authenticate repeatedly and across many systems, so without inventory, ownership, and expected behaviour baselines, the same telemetry can look harmless or hostile depending on context. That gap turns NHI visibility into an incident-response blind spot.

Q: How do teams know whether a directory change is persistence or approved administration?

A: They need lifecycle evidence tied to the change, not just the change record itself. If the environment cannot show who approved it, why it happened, and whether it matches expected operations, analysts have no reliable basis for classification. Persistence investigations become far more accurate when governance records are queryable alongside directory telemetry.

Q: Should security teams prioritise service-account visibility or broader detection tuning first?

A: Service-account visibility should come first when the main risk is credential theft or ambiguous authentication traffic. You cannot tune detections well if you do not know which identities are non-human, who owns them, and what normal looks like. Once that baseline exists, broader detection tuning becomes faster and less noisy.

Technical breakdown

Why password spray is hard to separate from normal identity noise

Password spray looks simple in principle, but the detection problem is contextual. In directory telemetry, the same outward symptoms can arise from legitimate lockouts, password rotation waves, or approved bulk operations. The benchmark shows that without correlating authentication patterns to host, account, and timing context, even strong models struggle to distinguish a spray from benign noise. Identity-tier detection therefore depends on more than alert volume. It requires sequencing, baselining, and cross-source correlation across directory and endpoint data.

Practical implication: build detections that correlate authentication bursts with directory and endpoint context instead of relying on a single alert type.

How domain persistence appears in Windows directory telemetry

Domain persistence is often visible as account manipulation, lifecycle changes, or long-lived access that should not exist in the normal administration pattern. In a Windows estate, the key problem is distinguishing approved identity maintenance from malicious persistence through the directory itself. When the analyst cannot tie a change to an authorised lifecycle event, the signal is not simply the change, but the absence of governance evidence around it. That is why persistence investigations are often more tractable than sprays: the artefacts are richer, but the analyst must still separate legitimate administrative actions from abuse.

Practical implication: require lifecycle evidence for directory changes so persistence investigations can test authorisation, not just identify modification.

Service-credential theft depends on what the domain controller can actually prove

Service-credential theft on a domain controller is a non-human identity problem as much as a detection problem. Service identities often authenticate repeatedly, interact with multiple systems, and create traffic that looks routine unless the defender knows the expected service pattern. The benchmark’s detection task highlights a familiar governance issue: without visibility into which identities are service identities, defenders cannot isolate theft from normal service authentication. That is an NHI governance gap disguised as a telemetry problem.

Practical implication: inventory service identities and define their expected authentication patterns before writing detection logic.

Threat narrative

Attacker objective: The objective is to gain durable control over the identity tier so the attacker can move through the environment using trusted accounts and service credentials rather than noisy exploit paths.

1. Entry begins with identity-tier abuse such as password spray against Windows accounts, where the attacker tests weak or reused credentials until one account gives access.
2. Escalation follows through account manipulation or persistence in Active Directory, allowing the operator to retain access even when the initial compromise is noticed.
3. Impact comes from credential theft or hands-on-keyboard control inside the directory environment, which lets the attacker operate as a trusted insider and reshape identity access at will.

NHI Mgmt Group analysis

Identity-tier attacks expose a governance blind spot, not just a detection gap. The benchmark shows that weak alerts on the directory edge are easy to misread when approved administration, lockouts, and service authentication generate similar signals. That means the real weakness is not only model performance but the absence of identity context that proves whether an action belongs to a normal lifecycle event. Practitioners should treat directory governance evidence as part of detection design, not a separate process.

Non-human identity visibility is the decisive control boundary in service-credential theft. Service identities are often the least visible accounts in the environment, yet they generate the most ambiguous telemetry. Without a clear inventory of service accounts, expected authentication cadence, and owner accountability, defenders cannot reliably distinguish theft from routine system behaviour. In NHI terms, this is a visibility and lifecycle failure, not merely an analytics problem.

Directory persistence succeeds when lifecycle assumptions are too broad. The benchmark’s domain-persistence case shows how easy it is to confuse approved changes with malicious state retention when governance records are incomplete or delayed. That is the same failure mode seen in real identity compromises: analysts assume access changes will be obvious, but the environment often treats them as normal administration. The practitioner lesson is to make lifecycle evidence machine-readable and queryable.

Detection engineering for Windows identity attacks needs a named concept: identity lookalike noise. This benchmark shows that benign administrative actions can resemble malicious activity closely enough to distort both hunting and incident response. The concept matters because the analyst is not just filtering false positives, but separating structurally similar identity events that differ only in authorisation and intent. Teams should design detections around this lookalike problem rather than around single-event heuristics.

Blue team maturity in identity telemetry now depends on correlation discipline. The strongest models in the benchmark did not win by spotting a single unusual event. They won by assembling host, account, directory, and authentication evidence into a consistent story. That same discipline is what IAM, IGA, and PAM teams need when they align privilege governance with detection, because the attack surface is the identity graph itself.

What this signals

Identity lookalike noise will keep shaping detection quality as long as teams treat directory telemetry as a flat alert stream rather than a governed identity graph. That means IAM, PAM, and detection engineering teams need shared baselines for approved administration, service-account behaviour, and lifecycle events, with evidence that can be queried during an investigation.

The programme implication is clear: if service identities are not visible, the detection layer is guessing. Teams should connect identity inventory, directory change provenance, and authentication analytics so that investigations can prove whether an event is normal, malicious, or simply unresolved.

A stronger operating model pairs governance and detection, not one after the other. When lifecycle state, ownership, and privilege scope are available in near real time, defenders can reduce false positives and accelerate response across both human and non-human identities.

For practitioners

• Correlate authentication bursts with lifecycle events Link password-spray alerts to password resets, bulk changes, and approved admin actions before escalating. That reduces false attribution from benign identity churn and makes the alert meaningful for incident response teams.
• Inventory service identities and owners Create a current catalogue of service accounts, their owners, and their expected authentication patterns. Use that inventory to separate routine service traffic from possible credential theft on domain controllers.
• Require change provenance for directory modifications Record the business or operational reason for every privileged identity change so investigators can verify whether a change was authorised or part of a persistence chain.
• Tune detections against benign lookalikes Test hunt and detection logic against password rotation waves, approved host changes, and other legitimate administrative patterns so the control fails safe before attackers do.

Key takeaways

• The benchmark shows that Windows identity attacks are hard to classify when benign administration looks similar to compromise.
• The strongest results came from models that correlated identity, endpoint, and directory evidence instead of relying on isolated alerts.
• Security teams need service-account visibility and lifecycle provenance before detection tuning can reliably separate abuse from routine activity.

Key terms

• Identity Lookalike Noise: Identity lookalike noise is benign administrative activity that resembles malicious behaviour closely enough to confuse alerting and investigation. It commonly appears in directory telemetry as password rotations, bulk account changes, approved host updates, or repeated service authentication that must be separated from abuse through context and provenance.
• Directory Persistence: Directory persistence is a technique for maintaining long-term access by modifying identity infrastructure rather than relying on a single compromised account. In Active Directory environments, attackers may alter objects, permissions, or replication-related settings so access survives password resets, account lockouts, or routine monitoring.
• Service-credential Theft: Service-credential theft is the compromise of non-human identities such as service accounts or tokens used by applications and infrastructure. It is especially difficult to spot because those identities authenticate repeatedly and can blend into normal workload traffic unless teams know the expected behaviour and ownership of each account.
• Access Provenance: Access provenance is the record of how an identity was created, approved, used, and withdrawn. In NHI governance, it is the evidence trail that lets teams prove an account is legitimate, explainable, and still within its intended access boundary.

What's in the full report

Cotool's full analysis covers the experimental setup and scoring detail this post intentionally leaves for the source:

• Per-task scoring breakdowns across the four simulated identity attacks, including where each model gained or lost points.
• Model-by-model cost and latency comparisons for the 168 total runs, useful if you are evaluating operational trade-offs.
• Methodology notes on how the hidden ground truth, red herrings, and deterministic detection scoring were constructed.
• Task design details for the password spray hunt, domain persistence investigation, service-credential theft detection, and hands-on-keyboard case.

👉 Cotool's full benchmark write-up includes task design, scoring methodology, and model-by-model cost and latency results.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, IAM, and secrets management. It helps practitioners connect identity controls to the operational realities of detection, lifecycle management, and privilege risk.