Security teams should combine email authentication, strong access controls, and behavior-based monitoring with continuous user coaching. AI makes phishing more convincing, so defenders need layered controls that reduce spoofing, detect unusual requests, and interrupt risky actions early. The goal is not just to block messages, but to lower the chance that a personalized lure reaches a high-value user and succeeds.
Why This Matters for Security Teams
Spear phishing becomes materially harder to spot when generative AI can tailor tone, timing, job role, and relationship cues at scale. That changes the defender’s job from spotting obvious fraud to interrupting convincing requests before they reach a user with access. Controls such as email authentication, payment verification, and privileged workflow checks still matter, but they need to be paired with identity-aware detection and process friction for sensitive actions. Current guidance suggests treating AI-personalised lures as a social engineering escalation, not a separate threat class.
The operational risk is highest where attackers combine public information, compromised mailbox context, and fast-moving impersonation across email, chat, and collaboration tools. The best defensive lens is a blend of resilience and verification, informed by resources such as the CISA cyber threat advisories and technique mapping from the MITRE ATT&CK Enterprise Matrix. In practice, many security teams encounter the failure only after a trusted user has already approved a fraudulent request, rather than through intentional resistance testing.
How It Works in Practice
Defence works best when the email layer, identity layer, and human verification layer are aligned. Email authentication reduces direct spoofing, but it does not stop a convincing message sent from a compromised account or a lookalike domain. Behaviour-based monitoring helps identify unusual sending patterns, impossible travel, first-time recipients, and atypical approval chains. For higher-risk workflows, teams should require step-up verification outside the original channel, especially for payment instructions, credential resets, gift card requests, payroll changes, and external file-sharing approvals.
AI-driven lures also create a detection problem because the content can look normal while the context is suspicious. That is why alerting should focus on identity and intent signals, not just text similarity. Mapping internal controls to NIST SP 800-53 Rev 5 Security and Privacy Controls helps teams formalise controls around access enforcement, auditability, and user awareness. If generative AI is also being used internally for drafting or triage, then the NIST AI 600-1 GenAI Profile is useful for reducing prompt injection risk, output misuse, and weak governance around AI-assisted communications.
- Authenticate messages, then verify intent for any sensitive request.
- Monitor for sender compromise, unusual session behaviour, and mailbox rule changes.
- Use short, role-specific coaching that reflects current lures, not generic awareness slides.
- Require second-channel confirmation for financial, credential, and access changes.
- Route high-risk cases to SOC or fraud review when identity confidence drops.
These controls tend to break down in organisations with flat approval processes, high-trust executive exceptions, and overreliance on email as the primary business system because attackers only need one unverified channel to succeed.
Common Variations and Edge Cases
Tighter verification often increases user friction and support overhead, so organisations must balance fraud reduction against business velocity. That tradeoff becomes sharper when executives, finance teams, recruiters, and help desks expect rapid responses. Current guidance suggests that the highest-friction controls should be reserved for the highest-impact actions, rather than applied uniformly to every message.
One edge case is the use of AI-generated lures inside collaboration tools or SMS, where traditional email filtering offers little protection. Another is multilingual phishing, where generative AI can localise language and cultural references well enough to bypass training that assumes obvious grammar errors. Teams should also consider whether external attackers are using AI to support reconnaissance, content adaptation, and follow-up persistence, which is where the MITRE ATLAS adversarial AI threat matrix is helpful for understanding attacker adaptation patterns. The Anthropic report on the first AI-orchestrated cyber espionage campaign is a useful reminder that AI can compress attacker effort, even when the underlying goals remain familiar.
There is no universal standard for user coaching cadence, but best practice is evolving toward targeted simulations, just-in-time warnings, and controls that trigger when identity confidence or request context changes unexpectedly.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK and MITRE ATLAS address the attack and risk surface, while NIST CSF 2.0, NIST AI RMF and NIST AI 600-1 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | PR.AA | Identity-aware access and awareness reduce success of personalized phishing. |
| MITRE ATT&CK | T1566 | Spear phishing is the core delivery technique behind AI-personalised lures. |
| NIST AI RMF | GOVERN | AI-enabled phishing changes model risk and requires governance over misuse patterns. |
| NIST AI 600-1 | GenAI profile helps manage misuse of generative tools in phishing and internal workflows. | |
| MITRE ATLAS | ATLAS captures adversarial AI tactics used to personalise and scale social engineering. |
Strengthen access, awareness, and monitoring controls around high-risk requests and accounts.
Related resources from NHI Mgmt Group
- How should security teams defend against AI-generated phishing at enterprise scale?
- How should security teams defend against AI-personalised phishing in email?
- How should security teams defend against TOAD phishing campaigns that use phone callbacks?
- How should security teams handle identity verification when attackers can use generative AI to spoof face, voice, and documents together?
