What breaks when detection rules assume attackers are either human or scripted?

Why This Matters for Security Teams

Detection logic built around a clean human-versus-script distinction misses the operating pattern that agentic activity now introduces. A human may type irregularly and pivot slowly, while a script often produces repetitive, machine-like bursts. An AI agent can combine both behaviours in one intrusion chain: it can wait, probe selectively, adapt to defences, then resume with new tool use or payloads. That makes simple thresholds, fixed timing assumptions, and rigid sequence rules far less reliable.

This is a practical problem for SOCs, cloud defenders, and threat hunters because many alerting rules still depend on dwell time, request rate, or a predictable order of events. The issue is not that those controls are useless, but that they were designed for older attacker models. Current guidance suggests defenders should interpret behaviour in context, using a broader telemetry picture rather than a single signal. The MITRE ATT&CK Enterprise Matrix remains useful here because it maps adversary behaviour as techniques, not just tooling.

In practice, many security teams encounter this failure only after an attacker has already blended low-and-slow activity with short automated bursts, rather than through intentional detection design.

How It Works in Practice

Effective detection has to move from static “human or script” logic to behaviour chains, context, and control-plane signals. An agentic intrusion can look like a legitimate user session at one moment and an automated operator at the next. That means defenders should combine identity, endpoint, network, and application telemetry, then correlate action quality, not just action count. The goal is to identify capability changes, unusual tool selection, and suspicious task progression.

A practical detection model usually includes:

• Session continuity checks that look for improbable pauses, resumptions, or task switching inside the same identity or token.
• Sequence analysis that detects when reconnaissance, privilege escalation, and lateral movement appear in an unusual order or with inconsistent pacing.
• Identity and access telemetry that flags token reuse, abnormal API invocation, or sudden privilege expansion.
• Content and output validation that checks whether AI-generated actions are being used to drive external systems without adequate review.

Security teams can map these patterns to existing control sets rather than inventing everything from scratch. The NIST Cybersecurity Framework 2.0 helps structure detection, response, and recovery, while the NIST SP 800-53 Rev 5 Security and Privacy Controls provides control language for audit logging, continuous monitoring, and incident response.

For AI-specific attack patterns, defenders should also review the MITRE ATLAS adversarial AI threat matrix alongside incident intelligence from CISA cyber threat advisories. These sources help distinguish model-driven abuse, orchestration abuse, and classic intrusion activity. These controls tend to break down when telemetry is sparse, tokenised service traffic is unattributed, and defenders cannot link individual actions to a stable identity or workload context.

Common Variations and Edge Cases

Tighter behavioural detection often increases false positives and tuning overhead, requiring organisations to balance sensitivity against analyst capacity. That tradeoff becomes sharper in environments where automation is already normal, such as CI/CD pipelines, API-heavy SaaS estates, and shared service accounts. In those settings, an attacker may look “scripted” even when the activity is agent-driven, so rigid rules can over-alert on legitimate automation while still missing adaptive abuse.

There is no universal standard for this yet, but current guidance suggests using layered detection rather than a single classification rule. One useful approach is to treat “human,” “script,” and “agent” as operational hypotheses, then test them against timing, intent, tool choice, and downstream effects. That is especially important where an AI system can trigger external actions, because the security question is no longer just who typed the command, but who or what had execution authority.

For teams with mature analytics, anomaly detection should be paired with allowlists, identity assurance, and response playbooks that account for mixed-mode behaviour. The Anthropic report on the Anthropic — first AI-orchestrated cyber espionage campaign report is a useful reminder that AI-enabled operations can compress steps that defenders historically expected to be separated by human pacing. The edge case is highly orchestrated, low-noise activity in trusted environments, where detections fail because every individual event looks plausible on its own.

Related resources from NHI Mgmt Group

• What breaks when payment fraud controls assume a human is always the actor?
• What breaks when remediation still assumes human-paced attackers?
• What breaks when bot detection only looks for human versus automated traffic?
• What breaks when identity security depends only on new detection rules?