TL;DR: Generative AI is helping attackers produce convincing, typo-free phishing at scale, making employee inboxes a more reliable target and weakening the usual red flags defenders rely on, according to Abnormal AI’s webinar. The security shift is not just better lures, but faster, more accessible social engineering that forces email defence and identity controls to work together.
NHIMG editorial — here’s why we think this discussion matters
By the numbers:
- 96% of technology professionals identify AI agents as a growing security threat, and 66% believe this risk is immediate.
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%).
- When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes and as quickly as 9 minutes in some cases.
Questions worth separating out
Q: How should security teams respond to AI-generated phishing campaigns?
A: Security teams should assume the message quality will be good enough to fool users and focus on reducing what a successful click can do.
Q: Why do AI-generated phishing emails increase account takeover risk?
A: They increase account takeover risk because they remove many of the clues users once relied on to spot fraud, while allowing attackers to personalise messages at scale.
Practitioner guidance
- Tighten mailbox-to-identity escalation paths Require stronger verification before password resets, MFA re-enrolment, help-desk changes, or privileged account recovery when mailbox behaviour looks abnormal.
- Adopt phishing-resistant authentication Prioritise MFA methods that resist prompt abuse and token replay, especially for high-value users, administrators, and support functions.
- Connect email telemetry to IAM response Feed suspicious inbox events into identity workflows so risky message patterns can trigger conditional access tightening, session review, or forced reauthentication.
What to expect at the briefing
Abnormal AI's full webinar covers the operational detail this post intentionally leaves for the source:
- Examples of AI-generated phishing patterns and how they differ from legacy email fraud
- Guidance on using AI-powered detection to identify malicious message behaviour at scale
- Practical ways to combine email security with identity controls such as MFA and access governance
- The webinar framing from Abnormal AI's Field CISO on how defenders can respond to AI-powered threats
👉 Watch Abnormal AI's on-demand webinar on AI-generated phishing and defence →
AI-generated phishing is changing email risk for security teams?
Explore further
Generative AI has turned phishing from a craft problem into a scale problem. The core change is not sophistication alone, but the removal of attacker friction. When novice attackers can generate polished lures in volume, the limiting factor shifts from writing skill to targeting and operational follow-through. For practitioners, that means awareness training remains necessary but no longer sufficient as a primary control.
A few things that frame the scale:
- 96% of technology professionals identify AI agents as a growing security threat, and 66% believe this risk is immediate, according to AI Agents: The New Attack Surface report.
- Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
A question worth separating out:
Q: How can organisations tell whether their anti-phishing controls are actually working?
A: Look at whether suspicious sign-in attempts are blocked, whether recovery workflows require stronger verification, and whether compromised accounts can be contained before they are used for privilege escalation. If users can still reset access, approve changes, or reach sensitive systems after a phish, the control stack is too weak.
👉 Read our full editorial: Generative AI is making phishing harder to spot and defend