TL;DR: ChatGPT has lowered the labour cost of phishing and impersonation, making AI-assisted abuse easier to scale, according to Abnormal AI. Mail security and identity checks have to absorb faster iteration and more credible lures before trust turns into action.
Editorial analysis by NHI Mgmt Group, based on content published by Abnormal AI: “Facing Your Fears: How Attackers Can Use Generative AI”.
Key questions
Q: Why do personalised AI-generated lures increase security risk?
A: They reduce the signal gap between legitimate outreach and malicious outreach by using public data to create highly contextual messages at scale.
Q: How can teams reduce the impact of AI-driven impersonation attempts?
A: Teams should combine user verification, conditional access, and response playbooks that isolate suspicious activity quickly.
Practitioner guidance
- Harden identity verification for high-risk requests Require out-of-band confirmation for payment changes, account recovery, privilege requests and executive approvals when the request arrives through email or chat.
- Reduce reliance on text-only trust signals Treat message tone, grammar and polish as weak evidence of legitimacy and combine them with sender context, workflow context and known request patterns.
- Tune detections for AI-assisted impersonation Look for bursts of similar lures, rapid content variation, unusual sender behaviour and repeated targeting of the same business process.
Bottom line: ChatGPT lowers the labour cost of phishing and impersonation, which makes AI-assisted abuse easier to scale.
Glossary terms: AI-assisted phishing, Impersonation blast radius, Human-centric trust signal, Verification Path
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
ChatGPT has changed attacker economics more than attacker intent. The most important shift is not that criminals gained a new objective, but that they gained a cheaper way to produce convincing abuse at scale. That means security programmes built around scarce attacker effort now face a volume and variation problem that human review cannot absorb reliably.
A few things that frame the scale:
- More than 80% of enterprises will have used generative AI APIs or deployed GenAI applications by 2026, up from 5% in 2023, according to Gartner.
A question worth separating out:
Q: How should organisations respond to AI-generated election impersonation?
A: They should create a verification workflow that combines content provenance checks, authoritative source validation, and rapid public correction. The goal is to confirm whether a voice, video, or message is authentic before it shapes voter behaviour. Election teams need named owners, escalation paths, and pre-approved messaging so response is fast enough to matter.
👉 Read our full editorial: ChatGPT-era attack automation and the new cyber threat curve