How should security teams defend against AI-personalised phishing in email?

They should combine content inspection with behavioural and identity signals, because AI-personalised phishing is designed to look relevant, timely, and low-risk. The best defence is not a better spam rule, but correlation across sender reputation, account behaviour, and the user’s normal communication patterns so suspicious messages are flagged before action is taken.

Why This Matters for Security Teams

AI-personalised phishing changes the defender’s problem from “is this message malicious?” to “does this message fit the recipient’s context well enough to trigger trust?” That shift matters because attackers can now tailor tone, timing, references, and workflow language at scale, making simple keyword filters and generic spam scoring less reliable. Current guidance from CISA cyber threat advisories and NHI-focused research such as The State of Non-Human Identity Security shows that identity-aware detection is now a baseline requirement, not a niche control.

The operational risk is not limited to credential theft. A well-personalised phish can push a user toward approving MFA, authorising a malicious app, sharing a one-time code, or moving an internal conversation into a compromised channel. That is why teams need to correlate content with sender behaviour, account history, and unusual request patterns rather than treating email inspection as a standalone gate. In practice, many security teams encounter the first successful compromise only after the message has already appeared “plausible” to the target.

How It Works in Practice

Defence works best when email security, identity telemetry, and user context are evaluated together. The content layer still matters: it should inspect language, links, domain lookalikes, reply-chain manipulation, and brand impersonation. But for AI-personalised phishing, the stronger signal is whether the message matches normal communication behaviour for that sender, that recipient, and that business process. Teams should look for deviations in thread length, request urgency, attachment type, payment language, MFA prompts, and changes in tone that do not fit the relationship history.

Practically, this means building detection around correlated signals such as sender reputation, account age, mailbox rules, impossible travel, forwarding changes, and first-seen external contacts. The control set should also include user-risk scoring and step-up verification when a message asks for action that is unusual for the recipient. For cases involving OAuth abuse or cloud app impersonation, the issue is not just email content but identity and consent abuse, which is why NHI research such as The State of Non-Human Identity Security is relevant to email defence as well.

• Flag messages that combine high personalization with a first-time sender or a newly registered domain.
• Correlate email events with IAM, MFA, and mailbox audit logs before allowing risky actions.
• Require out-of-band verification for payment, credential reset, and vendor bank-change requests.
• Quarantine or annotate messages that reference internal projects, org charts, or recent events in a way that looks scraped or stitched together.

There is no universal standard for this yet, but current guidance suggests combining policy-based routing, detection engineering, and user verification rather than relying on one model score. These controls tend to break down in high-volume shared inboxes and fast-moving sales or support environments because legitimate urgency and unusual phrasing are common there.

Common Variations and Edge Cases

Tighter email screening often increases false positives, requiring organisations to balance user friction against the risk of approving a convincing lure. That tradeoff is especially visible in executive inboxes, procurement, finance, and customer-facing teams, where legitimate messages can resemble attack patterns. One useful exception is internal impersonation: AI-generated phishing from a compromised employee account may bypass content filters entirely, so sender trust cannot be treated as proof of safety.

Another edge case is conversation hijacking. If attackers reply inside a real thread, the message may inherit trusted context while still carrying malicious instructions. In those cases, message origin, thread integrity, and recent account changes matter more than style or spelling. Teams should also be cautious with AI-generated summaries and auto-replies, because those systems can accidentally reinforce the attacker’s framing if they ingest the compromised thread. For broader attack pattern tracking, the DeepSeek breach is a useful reminder that trust in AI outputs can be exploited at the point of interpretation, not only at delivery.

The practical answer is layered defence: content inspection, identity correlation, and process verification. Detection gets stronger when it is treated as a workflow problem rather than a message-scanning problem, and that is the direction reflected in CISA cyber threat advisories.

Related resources from NHI Mgmt Group

• How should security teams defend against AI-generated phishing at enterprise scale?
• How should security teams defend against modern email attacks that bypass legacy filters?
• How should security teams defend against phishing when attacks move beyond email?
• How should security teams respond to AI-assisted phishing and social engineering?