Should organisations change their response model because attackers are using LLMs?

Why This Matters for Security Teams

Attackers using large language models do not change the core intrusion lifecycle, but they do compress it. Reconnaissance can be automated across many targets, phishing content can be tailored at scale, and post-compromise actions can be scripted with less operator effort. That means defenders need faster triage, tighter identity controls, and containment decisions that happen earlier in the chain. Current guidance suggests treating AI-enabled attacks as an acceleration problem first, and an attribution problem second, because waiting for perfect certainty creates avoidable exposure.

The practical risk is not that every LLM-assisted campaign is novel, but that familiar techniques become cheaper, more adaptive, and harder to spot in time. Security leaders should map response playbooks to observable attacker behaviour rather than to the tooling used by the adversary. Frameworks such as the MITRE ATT&CK Enterprise Matrix remain useful because the techniques still resemble established access, execution, persistence, and impact patterns. In practice, many security teams encounter AI-assisted abuse only after identity misuse or lateral movement has already occurred, rather than through intentional early-stage detection.

How It Works in Practice

Operationally, organisations should update response models in three places: detection engineering, identity enforcement, and containment automation. LLM use by attackers often increases message volume, variation, and timing, so indicators that worked for manual campaigns may arrive too late. Defenders should therefore prioritise telemetry that reveals sequence, privilege change, and tool use, not just content anomalies.

A useful way to structure the response is:

• Harden identity paths so initial access is harder to scale, especially for email, VPN, cloud consoles, and privileged workflows.
• Use conditional access, MFA resistance, and least privilege to reduce the value of compromised credentials.
• Trigger triage on behavioural signals such as impossible travel, new device enrolment, unusual API activity, and abnormal privilege escalation.
• Automate safe containment steps like session revocation, token rotation, and account suspension when confidence thresholds are met.
• Feed response lessons into threat modelling using MITRE ATLAS adversarial AI threat matrix for AI-enabled abuse patterns and Anthropic — first AI-orchestrated cyber espionage campaign report for real-world tradecraft signals.

Security operations should also adjust escalation logic. When an LLM helps attackers draft lures, generate payload variants, or interact with stolen credentials, the defender’s window for manual review shrinks. That is where policy-backed automation becomes valuable: the goal is not full autonomy, but earlier action with clear rollback paths. Mature teams can align these playbooks with NIST SP 800-53 Rev 5 Security and Privacy Controls and broader response governance in the NIST AI Risk Management Framework where AI is part of the defensive stack. These controls tend to break down when telemetry is fragmented across cloud, identity, and endpoint tools because the response engine cannot correlate identity abuse quickly enough.

Common Variations and Edge Cases

Tighter response automation often increases false-positive handling and business disruption, requiring organisations to balance containment speed against operational friction. That tradeoff is especially important when the suspicious activity may be AI-assisted but still ambiguous. Best practice is evolving here, and there is no universal standard for how much automation should be used before analyst review.

Some environments need extra caution. In regulated sectors, aggressive containment can interrupt customer service, trading, or clinical workflows, so approval chains must be pre-approved and tested. In software-heavy environments, attacker use of LLMs may show up as source code tampering, dependency abuse, or prompt injection rather than classic malware delivery. In those cases, response needs to include software supply chain checks and prompt/output validation, as reflected in the OWASP Agentic AI Top 10 and the CSA MAESTRO agentic AI threat modeling framework. The emerging guidance is to treat AI-enabled attack paths as a force multiplier on existing intrusion methods, not as a separate class that replaces established response models. That matters most when organisations rely on slow approvals, weak identity proofing, or manual investigation queues because attackers can outpace human review before containment starts.

Related resources from NHI Mgmt Group

• How can organisations tell whether their current identity model still fits platform change?
• How do organisations stop a model’s safe response from becoming unsafe execution?
• What should organisations do before using AI to support incident response?
• When should organisations treat a model update as a security change?