AI phishing now bypasses signature-based email defenses

At a glance

What this is: NHI Mgmt Group's analysis, drawing on Abnormal AI's blog post: AI-generated phishing has shifted from labour-intensive spear phishing to fast, personalised attacks that evade signature-based email controls.

Why it matters: For IAM and security teams, this matters because email remains an identity attack surface, and the fallback of verifying requests by phone is weaker when deepfake audio and video can impersonate trusted people in real time.

By the numbers:

• AI now lets attackers compress 40 hours of target research, infrastructure build, and personalized email crafting into minutes on a laptop.
• One team cut phishing triage from 20 to 40 hours per week to 4 to 5 hours after deploying behavioral AI on top of an existing SEG.

Context

AI has changed phishing from a high-effort social engineering campaign into a low-friction identity attack that can be personalised at scale. The practical gap is not just detection quality, but the assumption that a suspicious email will still contain obvious artefacts such as a bad link, a malicious attachment, or a known signature.

This matters to IAM because email compromise is often the first step in account takeover, fraudulent payment requests, or delegated access abuse. If the message looks like normal business correspondence and the voice check can be spoofed, controls that depend on recognising known indicators lose much of their value.

The article also shows why behavioural detection is becoming a governance issue, not just a security tooling choice. Teams need controls that baseline normal communication patterns per user and per relationship, because the attacker’s objective is to blend into legitimate workflow rather than trigger traditional alerting.

Key questions

Q: How should security teams implement AI-driven phishing detection across email, headers, links, and attachments?

A: Security teams should use a multi-stage workflow that separates header analysis, URL and attachment scanning, screenshot review, and content classification. This reduces blind spots because phishing usually combines technical deception with social engineering. The key is to correlate signals, score risk contextually, and keep a human-readable verdict layer for escalation and response.

Q: Why do call-back checks fail against deepfake voice and video attacks?

A: Because the verification channel itself can now be spoofed in real time. If attackers can imitate a trusted person’s voice or image long enough to complete the exchange, a call-back no longer proves identity on its own. Teams need corroboration from separate workflow, device, or transaction signals.

Q: What are the signs that phishing controls are failing in the browser rather than in the email gateway?

A: A common sign is when suspicious messages are blocked but users still reach realistic-looking login pages and enter credentials. Another signal is when campaigns keep working after a site is already known, because deny-lists update too slowly. If attackers are using CAPTCHA, fake overlays, or cloned interfaces to bypass user judgment, browser-layer controls are not keeping pace.

Q: What should teams do when a trusted contact requests urgent payment or access by email?

A: Treat the request as a workflow exception, not as a routine message. Require corroboration through an independent channel, check whether the request matches prior behaviour, and route anything unusual into fraud or identity review before approval is granted.

Technical breakdown

Why signature-based email security misses AI phishing

Signature-based email gateways and secure email gateways were built to inspect observable malicious artefacts. That model works when attacks rely on known indicators, suspicious URLs, or obvious payloads. AI-generated phishing often bypasses that layer because the message itself is syntactically clean, contextually plausible, and built from public or observed information rather than reused infrastructure. In identity terms, the attack abuses trust in the sender relationship, not in a malware sample. That makes traditional detection look for the wrong object. The deeper problem is that the threat is now behavioural and contextual, while the control still assumes a static malicious signature will exist.

Practical implication: move email defence logic beyond indicator matching and into sender-behaviour and conversation-context analysis.

How real-time deepfakes weaken call-back verification

Call-back verification is a compensating control that assumes a human can re-establish trust outside the email channel. Real-time audio and video deepfakes break that assumption by making the secondary channel impersonable on demand. That changes the attack from message deception to live identity impersonation, where the attacker can sustain a believable interaction long enough to authorise fraud or transfer decisions. The governance failure is not that teams lack a phone number to call, but that they treat voice or video as an inherently higher-trust confirmation channel. Once those signals are synthetic, the fallback loses its assurance value.

Practical implication: treat voice and video confirmation as a signal to corroborate, not as proof of identity on its own.

Why behavioural AI works better than prior-threat intelligence

Behavioural AI changes the detection problem from asking whether a message matches known bad content to asking whether the communication fits the normal pattern for that user, sender, or relationship. That is materially different from threat-intel-led filtering because it does not require prior knowledge of the attacker’s infrastructure or the exact lure. In practice, the model learns normal language, timing, counterparties, and transactional behaviour, then flags anomalies. This is why it can reduce triage load after deployment: many legitimate-looking messages are filtered out as normal, while out-of-pattern requests rise for review. The key is that the baseline must be specific enough to distinguish routine business from impersonation.

Practical implication: tune detection around behavioural baselines for users and business relationships rather than relying on static blacklists.

Threat narrative

Attacker objective: The attacker wants to convert believable impersonation into a trusted business action such as payment, account access, or delegated approval.

1. Entry begins with AI-assisted target research that quickly maps relationships, language patterns, and likely business workflows.
2. Credential or approval abuse follows when the attacker sends a personalised message that resembles a legitimate request and bypasses artefact-based email filtering.
3. Escalation occurs when the same campaign uses real-time voice or video spoofing to reinforce trust and pressure a payment or access decision.
4. Impact is achieved through fraudulent transfer, account compromise, or another identity-mediated business action that looks routine to the recipient.

Breaches seen in the wild

• Arup deepfake fraud 2024: Deepfakes of Arup's CFO and colleagues on a video call led a Hong Kong employee to transfer HK$200 million (about US$25.6m) to fraudsters.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 150+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI phishing turns email from a content-filtering problem into an identity-verification problem: the decisive issue is no longer whether a message contains a bad artefact, but whether the sender relationship and request pattern are authentic. Traditional SEG design was optimised for signatures, links, and attachments, which means it now sits one layer too low in the attack chain. Practitioners should treat conversational legitimacy as the control boundary, not message cleanliness.

Call-back verification has lost assurance value as a standalone fallback: voice and video used to provide a separate trust channel, but deepfake tooling collapses that assumption. This is a broader governance shift, because many organisations still rely on human recognition of tone, face, or familiarity as an implicit step-up check. The implication is that request verification must be designed as a multi-signal control, not a trust shortcut.

Behavioural baselining is becoming the practical control primitive for phishing defence: the strongest signal is often not a malicious indicator but a request that is out of character for the user, sender, or workflow. That shifts emphasis from threat intelligence to relationship intelligence, and from blocklists to anomaly detection. The control question is whether teams can distinguish normal business variation from impersonation without drowning reviewers in noise.

Identity blast radius now includes the inbox, the phone call, and the video call: phishing campaigns no longer need to stay inside email to succeed. Once attackers can move seamlessly across channels, the security programme has to govern the whole trust path that leads from message receipt to approval. The practitioner conclusion is simple: defend the decision, not just the mailbox.

Deepfake impersonation is now a routine fraud enabler, not a niche edge case: the article’s examples show how quickly synthetic media can be inserted into ordinary business workflows. That means identity assurance teams, fraud teams, and email security teams are now dealing with the same trust failure from different angles. The practical takeaway is to align those controls around one common verification model.

What this signals

AI phishing is forcing a shift from message security to trust-path security: practitioners should assume that clean-looking email can still be hostile, and design controls around sender behaviour, workflow context, and approval risk instead of artefact scanning alone.

Deepfake impersonation widens the attack surface beyond the inbox: when voice and video can be synthesised on demand, the identity programme has to govern how trust is established across channels, not just how messages are filtered.

Behavioural baselining changes the operational burden of phishing defence: teams that model normal communication patterns can reduce manual triage and focus reviewers on exceptions that truly deviate from expected business behaviour.

For practitioners

• Strengthen behavioural email detection Baseline normal sender behaviour, conversation timing, and relationship patterns so out-of-character requests surface even when the message is technically clean.
• Reduce reliance on call-back verification Require a second corroborating factor for urgent payment or access requests, because real-time voice and video can now be spoofed convincingly.
• Classify and escalate impersonation attempts Route suspicious business requests into fraud and identity review workflows instead of treating them only as email hygiene issues.
• Measure triage load and false positives Track how many phishing reports still require human review after behavioural controls are added, then use that signal to tune baselines.

Key takeaways

• AI-generated phishing now succeeds by looking like ordinary business communication rather than by carrying obvious malware indicators.
• Real-time voice and video spoofing weakens the classic call-back verification fallback that many teams still trust.
• Behavioural detection and relationship baselining are becoming the practical controls that matter most when signature-based email defence falls short.

Key terms

• AI Phishing: AI phishing is the use of generative AI to create highly convincing phishing messages, websites, audio, or video at scale. The content is often personalized, fluent, and context-aware, which makes visual inspection less reliable and pushes defenders toward behavioral analysis, identity context, and threat intelligence.
• Behavior Baseline: A record of normal activity for a non-human identity, including typical consumers, resources, and actions over time. Baselines help security teams detect when an identity is being used in an unusual way and provide the context needed to enforce least privilege safely in dynamic environments.
• Deepfake-based impersonation: A fraud technique that uses synthetic audio, video, or both to make an attacker appear to be a trusted person during a live interaction. The tactic exploits human trust in familiar cues and often aims to trigger urgent actions such as payments, resets, or access changes before verification is challenged.
• Secure Email Gateway: A secure email gateway is a control layer that inspects email before it reaches users and can also inspect outbound mail. It filters malicious content, enforces policy, and reduces exposure to phishing, malware, and data leakage, but it does not replace identity governance or account monitoring.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.