They should prioritise controls that evaluate behaviour in near real time, not just known malicious indicators after the fact. If campaign mutation outpaces analyst review, detection must shift toward baselines, contextual scoring, and automated correlation so one changed message does not become a missed intrusion. Manual review still matters, but it can no longer be the primary gate.
Why This Matters for Security Teams
AI-powered phishing no longer behaves like a static email threat. Messages can be rewritten, repackaged, and re-sent faster than an analyst can triage them, which means signature-based filters and queue-driven review only catch yesterday’s version of the attack. Security teams need controls that evaluate message context, sender behaviour, infrastructure reputation, and user-risk signals in near real time, consistent with the direction of the NIST Cybersecurity Framework 2.0. That shift matters because AI-assisted lures can adapt to defender feedback almost immediately, making manual validation too slow to serve as the primary control.
The operational risk is not just delivery of a malicious link. A fast-mutating phishing campaign can be used to capture session tokens, hijack SaaS access, or trigger follow-on fraud before human review closes the loop. The compromise often starts with one convincing message and then spreads through identity abuse, not malware. NHIMG’s research on the State of Non-Human Identity Security shows how quickly credential abuse becomes a broader security issue when monitoring is weak and access is over-privileged. In practice, many security teams discover this after one changed lure has already bypassed review and reached a high-value user, rather than through intentional testing of adaptive phishing controls.
How It Works in Practice
The practical response is to move from content review to behavioural decisioning. Instead of asking whether a message matches a known phishing template, teams should ask whether the message and its surrounding signals fit a trustworthy pattern for that sender, that tenant, and that user at that moment. This means combining email security telemetry, identity logs, endpoint signals, URL analysis, and sandbox or detonation results into a single risk score that can be enforced automatically.
A useful operating model includes:
- Real-time scoring of sender reputation, domain age, lookalike indicators, and delivery anomalies.
- Context-aware checks for impossible travel, unusual login prompts, token theft indicators, and anomalous OAuth consent activity.
- Automated quarantine, bannering, or detonation for messages that exceed a risk threshold.
- Rapid correlation with identity and endpoint telemetry so one message can be linked to downstream sign-in abuse.
- Post-delivery monitoring for user interaction, credential submission, and session token misuse.
This is especially important for AI-generated phishing because the attacker can vary wording, tone, and formatting without changing intent. The defender therefore has to evaluate the surrounding pattern, not just the text. Guidance from the DeepSeek breach reinforces a broader point: once credentials or systems are exposed, attacker follow-up can happen quickly, so response windows must be measured in minutes, not review cycles. Current practice also aligns with the identity-centered approach in NIST CSF 2.0, which treats detection and response as continuous functions rather than isolated checklist steps. These controls tend to break down when mail flows are highly fragmented across legacy gateways, isolated SaaS tenants, and manual exception lists because correlated telemetry cannot be evaluated at speed.
Common Variations and Edge Cases
Tighter automated screening often increases false positives and helpdesk friction, so organisations must balance blocking speed against operational disruption. That tradeoff is especially visible in executive mailboxes, finance workflows, and external partner communications where a slightly unusual message may still be legitimate. Current guidance suggests using adaptive thresholds rather than a single hard rule across all users, because one-size-fits-all controls can either miss targeted attacks or over-quarantine business-critical mail.
There is also no universal standard for how much human review should remain in the loop. Best practice is evolving toward human oversight for ambiguous cases and automated action for high-confidence malicious patterns. In environments with heavy use of shared inboxes, delegated access, or OAuth-connected productivity tools, phishing detection should also include identity-centric checks, not just email content inspection. The State of Non-Human Identity Security highlights why this matters: poor monitoring and over-privilege make a single successful lure far more damaging than the message itself. For teams that need a standards anchor, the NIST Cybersecurity Framework 2.0 supports this shift from static review to continuous protection. The model becomes fragile when organisations rely on manual approvals for high-volume mail streams because the review process cannot keep pace with attacker mutation.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST CSF 2.0 and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | DE.CM-1 | Continuous monitoring is required when phishing mutates faster than review. |
| OWASP Non-Human Identity Top 10 | NHI-03 | Phishing often leads to stolen credentials and token misuse against NHIs. |
| NIST AI RMF | AI RMF fits adaptive phishing because risk must be assessed dynamically. |
Use AI RMF to govern continuous risk scoring, oversight, and response for adaptive threats.
Related resources from NHI Mgmt Group
- How should security teams handle modern phishing when attackers spoof trusted roles?
- How do teams know whether their email security controls are keeping up with AI phishing?
- How should security teams handle phishing messages that auto-forward into business apps?
- What steps should security teams take to prevent Shadow AI risks?
