Generative AI is reshaping cybercrime tactics and attack success

At a glance

What this is: This on-demand webinar argues that generative AI is materially changing how cybercriminals plan and execute attacks, with attacker tooling and success rates both evolving fast.

Why it matters: It matters to IAM and security teams because AI-assisted social engineering, token abuse, and identity compromise can outpace controls built for human-paced threat activity.

👉 Watch Abnormal AI’s on-demand webinar on generative AI and cybercrime

Context

Generative AI is changing cybercrime by lowering the effort needed to create convincing lures, automate variants, and test attacks at scale. The identity angle is that these campaigns do not stay purely in the malware layer. They increasingly target people, sessions, tokens, and the trust decisions that IAM and security operations rely on.

For practitioners, the key question is not whether attackers use AI, but which parts of the attack chain become cheaper and faster when they do. That shifts the burden onto detection, identity assurance, and response processes that can keep up with adaptive, higher-volume abuse.

Key questions

Q: How should security teams respond to AI-assisted phishing and social engineering?

A: Treat AI-assisted phishing as a scale and quality problem, not just a messaging problem. Tighten authentication at the point of approval, train users on high-risk workflows such as payment and recovery, and monitor sessions for abnormal behaviour after credentials are entered. The goal is to make the attacker’s next step harder even if the lure succeeds.

Q: Why do generative AI threats matter to identity security teams?

A: Because most AI-assisted attacks still need identity to convert persuasion into impact. The attacker may use better language or higher campaign volume, but the final objective is usually a credential, token, or privileged action. That means IAM, PAM, and session monitoring are central to limiting the damage.

Q: What do teams get wrong about malicious LLMs like WormGPT?

A: They often focus on the model itself instead of the operational effect. The risk is not the label on the tool, but the way it lowers friction for phishing, fraud, and pretexting at scale. Security teams should evaluate how quickly those campaigns could reach their identity controls, not whether the model is famous.

Q: How can organisations reduce the impact of AI-enabled cybercrime?

A: Shorten the path from suspicious activity to identity containment. Use stronger authentication, narrow privilege, and rapid session review so a successful lure does not automatically become a successful breach. When attackers move faster, response has to be anchored in the identity layer, not only in email filtering or awareness training.

Background and context

How generative AI changes attacker tradecraft

Generative AI reduces the time cost of producing phishing content, adapting lures to different targets, and iterating messages after failure. In practice, that means attackers can test more variations, improve language quality, and localise campaigns with far less manual effort. The result is not a new class of attack so much as a scale and speed multiplier across familiar tactics such as credential theft, business email compromise, and social engineering. The threat is especially relevant where identity decisions depend on human judgment or static filters.

Practical implication: tune detection and awareness controls for rapid campaign variation, not just known-bad templates.

Why malicious LLMs raise the ceiling for abuse

Malicious LLMs such as WormGPT represent a shift from general-purpose AI use to purpose-built attacker tooling. These systems can be used to generate phishing content, refine fraud scripts, and support more persuasive pretexting without the guardrails that mainstream models apply. That matters because the attacker no longer needs to work around model safety barriers to obtain usable content. Even when the underlying attack remains simple, the quality and volume of output can improve enough to increase success rates and overwhelm static controls.

Practical implication: treat malicious AI tooling as an enabler of campaign volume and persuasion, not as a niche curiosity.

Why identity controls become the choke point

When AI increases attack success, identity becomes the most valuable control plane because many attacks still need a human to approve, a token to steal, or a session to hijack. That is where authentication strength, conditional access, session monitoring, and privileged access controls matter most. AI does not remove the need for identity compromise. It makes compromise more scalable and the pretexting phase more effective, which means identity telemetry and response need to be fast enough to catch abuse before access is converted into impact.

Practical implication: align phishing resistance, session telemetry, and privileged access monitoring around the points where attacker persuasion turns into access.

NHI Mgmt Group analysis

Generative AI is now an attack multiplier, not just a content tool. The practical change is not that cybercrime became novel, but that existing social engineering, fraud, and credential theft workflows became faster, cheaper, and more adaptable. That shifts defender assumptions about campaign volume, iteration speed, and the half-life of a lure. Practitioners should plan for adversaries who can generate and test many more variants before a human analyst ever sees them.

Malicious LLMs expose the weak point in human-paced security models. WormGPT and similar tooling matter because they strip away the safety friction that would otherwise slow offensive use of generative AI. The field implication is that static awareness training and signature-led detection no longer describe the real operating tempo of modern abuse. Practitioners should re-evaluate where human judgement is still the control and where machine speed has already overtaken it.

Identity is where AI-enabled crime becomes measurable. Even when generative AI improves the lure, the attacker still needs a credential, token, approval, or session to finish the job. That makes identity telemetry, privileged access governance, and fast containment the decisive control layer. Practitioners should treat AI-driven attack success as an identity problem once persuasion turns into access.

The concept is attack-surface compression. Generative AI compresses the time between campaign creation, target personalisation, and active abuse. That means defenders see less warning time and less reuse of the same malicious artefacts. The implication is that programme maturity now depends on reducing the time from first suspicious signal to identity containment, not on assuming attackers need long lead times.

From our research:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
• Only 44% of organisations have implemented any policies to govern AI agents, even though 92% agree that governance is critical to enterprise security.
• For a broader identity lens on where these risks sit in the attack surface, see The 52 NHI breaches Report for patterns that translate into machine identity and access failure modes.

What this signals

Attack-surface compression: generative AI shortens the time between lure creation, target-specific variation, and identity compromise. That means practitioners should expect more campaigns to reach the authentication layer before human defenders can adapt. The response is to compress containment time in parallel, especially where identity actions can be reversed before they spread.

With 80% of organisations already seeing AI agents act beyond intended scope in related research from AI Agents: The New Attack Surface report, the broader lesson is that autonomous or AI-assisted behaviour routinely escapes the boundaries designers expect. Security programmes should assume that speed and scope drift are now normal failure modes, not edge cases.

Security teams should map where AI-assisted abuse would first touch their identity controls, then design stronger checks for those exact handoff points. That includes inbox-to-approval transitions, help desk resets, privileged session issuance, and token use that looks valid but arrives from unusual behavioural context.

For practitioners

• Harden identity verification at the approval point Require phishing-resistant authentication and step-up checks wherever a message can trigger access, payment, or privilege changes. Focus on the exact moments where persuasion becomes an identity decision, especially in help desks, finance workflows, and privileged requests.
• Instrument session-level detection for token abuse Add monitoring for unusual session creation, impossible travel, token reuse, and sudden changes in inbox or identity-system behaviour. AI-assisted attacks often succeed by making the pretext better, so defenders need detection that starts after the credential or session is obtained.
• Limit blast radius for high-trust workflows Break sensitive workflows into smaller approval steps, shorten privilege duration, and require independent verification for account recovery, mailbox delegation, and admin elevation. This reduces the value of any single successful AI-assisted pretext.

Key takeaways

• Generative AI is accelerating familiar cybercrime techniques by making persuasion, iteration, and scale cheaper for attackers.
• The practical threat is identity-centred because successful AI-assisted attacks still need credentials, tokens, or approval to become impact.
• Teams should harden the approval layer, tighten privileged workflows, and detect abnormal session behaviour before persuasion turns into access.

Key terms

• Generative AI abuse: The use of text, code, or media generation systems to improve the speed, scale, or believability of offensive activity. In security practice, this usually means better phishing, more convincing pretexts, or faster campaign variation that increases the odds of identity compromise.
• Malicious LLM: A large language model used or adapted for offensive purposes, often to remove safety guardrails that restrict harmful output. For practitioners, the concern is not the label alone but the operational effect: more persuasive social engineering, faster iteration, and higher campaign throughput.
• Identity-centred attack: An attack that reaches impact by using credentials, sessions, approvals, or privileged workflows rather than only exploiting software code. This term matters because many AI-assisted campaigns succeed when a human or system grants access that the attacker then reuses.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or programme maturity, it is worth exploring.

This post draws on content published by Abnormal AI: AI, Cybersecurity, and the Promise of Tomorrow: A Hacker's Insight. Read the original.