Join our Newsletter — 33% off our NHI Course

Why is AI phishing making message checks less reliable?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 2 years ago
Posts: 20217
Topic starter  

TL;DR: AI-generated phishing now makes fluent language and personalization unreliable trust cues, ZeroFox reports, as grammar stops separating real from fake. Inbox controls now need sender checks and out-of-band verification because message quality no longer proves legitimacy.

Editorial analysis by NHI Mgmt Group, based on content published by ZeroFox: “AI Phishing Attacks: Why Bad Grammar Stopped Being a Red Flag”.

By the numbers:

  • 82.6 percent of phishing emails use AI-generated content, according to ZeroFox.
  • 57.9 percent of phishing attacks originate from compromised accounts, according to ZeroFox.
  • ZeroFox says the attacker cost drops by 95 percent when using LLMs.

Key questions

Q: Why do AI-powered email attacks increase risk even when staff are well trained?

A: AI powered attacks reduce the value of human judgment by making messages look routine, urgent, and personally relevant.

Q: How do you know whether phishing defence is working?

A: Look for three signals: faster time from report to containment, fewer repeat exposures from the same campaign, and better quality of employee reporting over time.

Q: How should security teams respond when phishing messages come from a legitimate, hijacked social account?

A: Treat the message as untrusted even if it comes from a known contact.

Practitioner guidance

  • Rebuild awareness around verification steps Train users to confirm payment changes, credential requests, and urgent executive instructions through a second independent channel, not by judging message quality.
  • Correlate infrastructure before trusting content Use domain age, registration patterns, campaign linkage, and sender reputation to score messages before they reach the user.
  • Extend phishing controls beyond email Include search results, social impersonation, fake job listings, and branded ad abuse in the same monitoring and takedown workflow.

Bottom line: AI-generated phishing removes the old reliance on poor grammar and awkward wording, so message appearance is no longer a dependable defence signal.

Glossary terms: AI Phishing, Lookalike Infrastructure, Out-Of-Band Verification, Phishing-as-a-service

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →


This topic was modified 6 days ago by NHI Mgmt Group

   
Quote
(@mr-nhi)
Member Moderator
Joined: 6 months ago
Posts: 20707
 

AI phishing is a governance problem, not a formatting problem. When attackers can generate fluent, context-aware messages at scale, the old reliance on linguistic tells becomes a brittle control. The real issue is that trust is being manufactured through content quality rather than technical legitimacy, so verification must move to identity, channel, and request validation. Security teams should treat message quality as noise, not evidence.

A few things that frame the scale:

  • The IBM/Ponemon 2025 Cost of a Data Breach Report found that phishing-initiated breaches cost an average of $4.8M each.

A question worth separating out:

Q: What is the difference between email security and phishing protection?

A: Email security focuses on the channel, while phishing protection has to cover the full abuse path, including lookalike domains, impersonation sites, social posts, and malicious ads. Many attacks now use email only as one delivery option, so a strong programme watches the brand and the request, not just the inbox.

👉 Read our full editorial: AI phishing is eroding the reliability of message-based defence


This post was modified 6 days ago by NHI Mgmt Group

   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.