Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

AI romance scams and deepfakes: are identity controls keeping up?


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 10745
Topic starter  

TL;DR: AI-generated personas can sustain convincing video calls, messages, and money requests for months, and Incode’s CEO says deepfakes are now a major attack vector with one-third of defended attacks involving impersonation. The real problem is not spotting the fake after the fact, but replacing instinct with verifiable identity controls before trust forms.

NHIMG editorial — based on content published by Incode: AI-powered romance scams and deepfake identity risk from The Perfect Scam episode

Questions worth separating out

Q: How should organisations handle romance scams that use deepfakes and AI agents?

A: Organisations should treat them as identity assurance failures, not just user awareness issues.

Q: Why do deepfakes create a bigger identity risk than ordinary phishing?

A: Deepfakes remove the visual and auditory cues people normally rely on to detect fraud.

Q: How can security teams reduce losses from AI-driven impersonation?

A: Focus controls on the moment of decision, not just the moment of first contact.

Practitioner guidance

  • Add step-up verification at money movement points Require stronger identity proofing before wire transfers, account recovery, beneficiary changes, or support-triggered privilege changes.
  • Limit trust escalation across channels Do not let a persuasive chat, video call, and phone call count as three separate proofs of the same identity.
  • Train support and finance teams on impersonation patterns Teach teams to treat urgency, secrecy, channel switching, and refusal of in-person or out-of-band verification as common fraud indicators.

What's in the full article

Incode's full post covers the operational detail this post intentionally leaves for the source:

  • The transcript of Ricardo Amper’s commentary on deepfake-driven romance scams and why traditional detection fails.
  • The real-world examples of impersonation attempts against finance and support workflows, including the treasury call incident.
  • The concrete verification practices discussed for platforms, social networks, and consumer identity flows.
  • The longer discussion of AI agents scaling fraud operations and what that means for future verification design.

👉 Read Incode’s discussion of AI-powered romance scams and deepfake identity risk →

AI romance scams and deepfakes: are identity controls keeping up?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 2 months ago
Posts: 10300
 

Deepfake romance scams are not a better phishing trick, they are an identity assurance failure. The attacker is no longer merely spoofing a message, they are sustaining a believable identity over time across video, voice, and chat. That means the control failure is not just user awareness, it is the absence of a stronger proofing and step-up model at the point where trust turns into money movement or recovery. Practitioners should treat this as an identity governance issue, not a content moderation problem.

A few things that frame the scale:

  • The average estimated time to remediate a leaked secret is 27 days, despite 75% of organisations expressing strong confidence in their secrets management capabilities, according to The State of Secrets in AppSec.
  • Only 44% of developers are reported to follow security best practices for secrets management, according to The State of Secrets in AppSec.

A question worth separating out:

Q: Who should be accountable when AI impersonation leads to fraud losses?

A: Accountability should sit with the teams that own identity proofing, support escalation, and financial approval workflows, because those are the points where trust becomes action. Fraud prevention cannot live only with the end user or only with security. It has to be shared across identity, operations, and finance governance.

👉 Read our full editorial: AI romance scams are scaling past human detection



   
ReplyQuote
Share: