Organisations should treat them as identity assurance failures, not just user awareness issues. Use stronger verification at high-risk moments, especially for money movement, recovery, and support escalation. The safest approach is to require independent proof before trust is converted into action, because deepfakes can now sustain convincing interactions long enough to bypass instinct.
Why This Matters for Security Teams
Romance scams powered by deepfakes and AI agents are not just fraud cases. They are identity assurance failures that exploit trust, persistence, and human escalation paths. A convincing voice clone, synthetic video, or scripted agent can maintain pressure long enough to move a victim from suspicion to action, especially when the scam is timed around payment, recovery, or support requests. That makes this a security and operations problem, not only a training problem.
Current guidance suggests treating the highest-risk moments as trust conversion points, where an organisation should require independent proof before allowing money movement or account recovery. That aligns with broader AI governance thinking in the NIST AI Risk Management Framework and the agent-focused controls discussed in OWASP Agentic AI Top 10. NHIMG research also shows the scale of the problem: in AI Agents: The New Attack Surface report, 80% of organisations said their AI agents had already performed actions beyond intended scope.
In practice, many security teams discover these scams only after a support desk, fraud team, or customer has already converted synthetic trust into a real transfer.
How It Works in Practice
The operational answer is to build friction around the points where deception becomes costly. That means using layered verification, not a single challenge. For example, if a person says they need urgent help, the organisation should route the case to a separate channel, use callback verification to a known number, and require proof that is hard to fake in real time, such as a previously established device, recorded passphrase, or in-app confirmation from a trusted session.
AI agents change the threat because they can sustain a conversation, adapt to objections, and generate different media on demand. Unlike a one-off phishing email, they can chain text, voice, and video into a coordinated manipulation flow. That is why static scripts and role-based approval trees are weak defenses here. Better practice is intent-based decisioning at the moment of action: what is being requested, who benefits, what account is involved, and whether the request matches prior behaviour. This is consistent with the direction of CSA MAESTRO agentic AI threat modeling framework and the identity controls described in Ultimate Guide to NHIs — 2025 Outlook and Predictions.
- Use step-up verification before account recovery, beneficiary changes, or refund redirection.
- Require out-of-band confirmation for high-value or unusual requests.
- Log and review deepfake indicators, but do not rely on them alone.
- Train support and fraud teams to treat urgency as a risk signal, not proof.
Where organisations need a deeper control model, they should also examine agent-adjacent attack paths such as support chatbot compromise in the Meta AI Instagram Account Takeover case. These controls tend to break down in high-volume contact centres where staff are measured on speed and consistency rather than verification quality.
Common Variations and Edge Cases
Tighter verification often increases friction and abandonment, so organisations must balance fraud reduction against customer experience and legitimate emergency use cases. That tradeoff is real, especially in financial services, healthcare, and support environments where urgency is common and attackers deliberately imitate distress. Best practice is evolving here, and there is no universal standard for when to require biometric checks, callback validation, or manual review.
Edge cases matter. A family member may truly need help, a victim may be locked out of an account, or a partner may be making a time-sensitive payment. The response should be proportional: escalate the level of proof as the value and reversibility of the action increase. For AI-generated media, the presence of a convincing face or voice should not be treated as strong identity evidence on its own. Organisations should prefer independent proof from known devices, prior enrolment, or verified channels, and they should coordinate fraud, support, and legal teams on a shared playbook. Threat research from the OWASP Top 10 for Agentic Applications 2026 and the MITRE ATLAS adversarial AI threat matrix both reinforce that dynamic, adaptive adversaries require context-aware defenses.
For organisations handling sensitive recovery flows, the key lesson is simple: if the request can move money, reset access, or override normal controls, treat the interaction as untrusted until independently verified.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, CSA MAESTRO and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A2 | AI agents and synthetic media are central to this scam pattern. |
| CSA MAESTRO | T2 | Supports threat modeling for adaptive, goal-driven AI abuse. |
| NIST AI RMF | GOVERN | Identity fraud with AI requires governance, accountability, and oversight. |
| OWASP Non-Human Identity Top 10 | NHI-01 | Synthetic identities and stolen secrets underpin account takeover paths. |
| NIST CSF 2.0 | PR.AC-1 | Verification and access control are the core defensive response here. |
Map agent abuse scenarios to A2 and require runtime checks before any action is executed.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on July 10, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
