Because they are better at mimicking normal language, timing, and reply behaviour than older filter logic expects. Secure email gateways still matter, but they are strongest against known patterns, not adaptive social engineering. The practical gap is identity context, since the real risk is often what the message causes a user or help desk to authorise.
Why This Matters for Security Teams
AI-generated business email compromise succeeds because it targets the control plane around trust, not just the inbox. Modern secure email gateway are good at matching known malicious signatures, spoofing cues, and commodity phishing patterns, but they are weaker when the message is syntactically normal, contextually relevant, and delivered in a believable sequence. That makes this a workflow and identity problem as much as a malware problem.
For NHI Management Group, the core issue is that AI-assisted BEC can adapt to the recipient’s role, recent projects, and escalation paths in ways that static filtering cannot reliably score. The right response is not to discard email defenses, but to pair them with stronger identity verification, transaction controls, and out-of-band approval rules. The threat profile is consistent with broader AI-enabled abuse patterns documented in the OWASP NHI Top 10 and observed in CISA cyber threat advisories.
In practice, many security teams encounter BEC only after a finance or help desk workflow has already legitimised the attacker’s request.
How It Works in Practice
AI-generated BEC bypasses traditional secure email gateways by removing the cues those systems rely on: poor grammar, repeated templates, obvious spoofing, and high-volume delivery patterns. The attacker can vary tone, thread structure, timing, and sender impersonation at scale, while preserving the kind of business language that looks ordinary to both humans and rules engines. Some campaigns are even built to continue a conversation over multiple replies, which makes them harder to flag as isolated malicious events.
The practical defense is layered and should move beyond message inspection alone. Current guidance suggests combining email controls with identity-aware verification, policy-based approval flows, and transaction-level checks for payment, credential resets, and vendor bank changes. The most effective programs also train staff to verify requests through a separate channel when the message creates urgency or bypasses normal process. For deeper background on identity-centric abuse, see the 52 NHI Breaches Analysis and the Ultimate Guide to NHIs — Key Challenges and Risks.
- Verify high-risk requests out of band, especially payments, gift card purchases, payroll updates, and password resets.
- Use allowlisted business processes, not just inbox filtering, to decide whether a request can proceed.
- Require step-up approval when a message changes account details, routing data, or supplier information.
- Log and correlate email events with identity and help desk activity to spot replayed or chained requests.
These controls tend to break down when approvals are informal and staff are conditioned to treat email as sufficient authority.
Common Variations and Edge Cases
Tighter verification often increases friction, so organisations must balance user convenience against the risk of social engineering. That tradeoff is especially visible in fast-moving finance, executive support, and outsourced service environments, where delays can disrupt legitimate work and encourage workarounds.
There is no universal standard for this yet, but current guidance suggests that the strongest controls are the ones that match the business action being requested. For example, a suspicious invoice request may need payment-system validation, while a password reset scam may need stronger help desk identity proofing and a short-lived approval token. Email gateway tuning still matters, but it should be treated as one signal, not the final decision. Where organisations adopt Anthropic’s AI-orchestrated cyber espionage report as a reference point, the key lesson is that language quality alone no longer separates benign from malicious content. The same is true in the DeepSeek breach context, where exposure and reuse of sensitive material accelerated abuse pathways.
These controls tend to break down in organisations that lack process ownership, because the attacker is then competing against human habit rather than a defined verification standard.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and CSA MAESTRO address the attack and risk surface, while NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A03 | AI-driven social engineering and tool abuse map to agentic app risk controls. |
| CSA MAESTRO | G1 | Governance for autonomous workflows is needed when AI shapes approval and reply paths. |
| NIST AI RMF | BEC is a trust and misuse risk that fits AI RMF governance and map functions. |
Document AI-related social engineering risks and define monitoring, escalation, and human oversight.
Related resources from NHI Mgmt Group
- How should security teams handle socially engineered email attacks that bypass secure email gateways?
- Why do traditional email gateways miss some advanced email attacks?
- Why do AI-generated email attacks increase identity risk?
- Why do AI-generated phishing emails weaken traditional email security models?