AI phishing is changing the detection gap, what should teams do now?

TL;DR: AI-generated phishing now accounts for 82.6 percent of phishing emails, according to ZeroFox, which means fluent language, personalization, and multilingual output no longer reliably signal legitimacy. Proof now has to move from message quality to infrastructure, sender reputation, and out-of-band verification because proofreading no longer catches the attack.

Editorial analysis by NHI Mgmt Group, based on content published by ZeroFox: “AI Phishing Attacks: Why Bad Grammar Stopped Being a Red Flag”.

By the numbers:

• 82.6 percent of phishing emails use AI-generated content, according to ZeroFox.
• 57.9 percent of phishing attacks originate from compromised accounts, according to ZeroFox.
• ZeroFox says the attacker cost drops by 95 percent when using LLMs.

Key questions

Q: How should security teams respond when phishing messages come from a legitimate, hijacked social account?

A: Treat the message as untrusted even if it comes from a known contact.

Q: Why do AI-powered email attacks increase risk even when staff are well trained?

A: AI powered attacks reduce the value of human judgment by making messages look routine, urgent, and personally relevant.

Q: How do you know whether phishing defence is working?

A: Look for three signals: faster time from report to containment, fewer repeat exposures from the same campaign, and better quality of employee reporting over time.

Practitioner guidance

• Rebuild awareness around verification steps Train users to confirm payment changes, credential requests, and urgent executive instructions through a second independent channel, not by judging message quality.
• Correlate infrastructure before trusting content Use domain age, registration patterns, campaign linkage, and sender reputation to score messages before they reach the user.
• Extend phishing controls beyond email Include search results, social impersonation, fake job listings, and branded ad abuse in the same monitoring and takedown workflow.

Bottom line: AI-generated phishing removes the old reliance on poor grammar and awkward wording, so message appearance is no longer a dependable defence signal.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →