AI-enabled romance scams expose the limits of trust controls

At a glance

What this is: This analysis argues that romance and investment scams are becoming more scalable through GenAI, platform migration, and coordinated money movement controls.

Why it matters: It matters to IAM practitioners because the same trust, verification, and lifecycle weaknesses that enable fraud also show where identity and access programmes fail to stop abuse across human, NHI, and platform-mediated interactions.

By the numbers:

• Over $10 billion in fraud and scam losses occurred in 2023.

👉 Read Arkose Labs' analysis of romance and investment scam controls

Context

Romance and investment scams are trust-exploitation campaigns that combine social engineering, platform abuse, and payment manipulation. The article’s core claim is that generative AI makes those campaigns cheaper to localise, more convincing to victims, and easier to sustain across text, social, dating, and messaging channels.

For IAM and identity governance teams, the lesson is not limited to fraud operations. These scams expose how weak identity assurance, poor account verification, and fragmented lifecycle controls allow hostile actors to impersonate legitimacy long enough to move the victim from initial contact to monetary loss.

Key questions

Q: How should organisations stop romance and investment scams before money moves?

A: Focus on the earliest controllable trust points. That means blocking suspicious entry messages, verifying high-risk profiles, slowing unusual transfers, and giving frontline teams clear escalation paths. The goal is to interrupt the scam before the victim is coached into private messaging or payment. The most effective programmes combine platform abuse signals with transaction-layer friction and human review.

Q: Why do GenAI-powered scams make traditional fraud controls less effective?

A: Because the attack now adapts in real time. GenAI can localise language, mirror tone, and create convincing visual or audio personas, which reduces the usefulness of static keyword filters and one-off review. Controls that only inspect a single message or a single channel miss the broader behavioural pattern that exposes the scam.

Q: What do security teams get wrong about scam prevention?

A: They often treat scams as a single-platform content problem. In reality, the attacker uses multiple identity layers, moving from contact channel to conversation channel to payment channel. If the programme only monitors one layer, the scam simply migrates to a less visible environment. Effective prevention follows the journey, not the initial message.

Q: Who should be accountable when a scam survives platform controls and reaches a bank transfer?

A: Accountability should be shared across the platform, the payments provider, and the operational team that approved the transfer. No single control failure explains the loss. If an organisation can identify fraudulent behaviour early but cannot coordinate action across channels, the scam will still reach settlement. Governance must match the whole attack path.

Technical breakdown

How GenAI changes scam credibility and scale

Generative AI lowers the cost of personalised deception. Scammers can translate messages, mimic fluent local language, generate talking-head videos, and adapt tone to the victim’s profile, which makes the interaction feel human and persistent. The important technical shift is not just content generation. It is the ability to maintain the appearance of continuity across channels while changing language, persona, and medium without losing coherence. That makes traditional keyword filters and one-channel detection brittle, because the attack is now multimodal and iterative rather than a single fraudulent message.

Practical implication: detection must look for behavioural consistency across channels, not just suspicious text strings in one channel.

Why platform migration weakens control points

The article describes a familiar scam progression: first contact by SMS, dating app, or social platform, then rapid migration to WhatsApp, Telegram, Signal, or similar tools where the conversation is harder to monitor. This creates a control gap because trust is established in one environment while monetisation happens in another. The governance problem is not only content moderation. It is identity verification across service boundaries, where the same actor can reappear with a new handle, new medium, and a stronger emotional claim to legitimacy. Once the scam leaves the original platform, intervention windows narrow sharply.

Practical implication: teams need identity verification and abuse-detection coverage before conversations can move into lower-visibility channels.

Why transaction-layer controls still matter

Even sophisticated manipulation still has to reach a money movement event. The article points to bank tellers, customer service staff, anomaly detection, and money mule controls as the final containment layer. This is where fraud shifts from persuasion to execution, and where contextual signals such as transaction size, destination, frequency, and customer behaviour become decisive. From an identity perspective, the challenge is that a trusted human can be socially engineered into authorising the wrong outcome. Controls therefore need to combine behavioural analytics, operational escalation, and account-level friction to slow or interrupt the transfer before funds exit.

Practical implication: strengthen transaction review and mule detection so suspicious transfers can be slowed before final settlement.

Threat narrative

Attacker objective: The attacker’s objective is to convert emotional trust into financial loss at scale by moving victims off monitored channels and into money transfer actions.

1. Entry begins with a mis-sent text message, social media contact, or dating-site approach that opens a trust-building conversation.
2. Escalation occurs as the scammer moves the victim onto encrypted or lower-visibility messaging platforms and uses GenAI to sustain a believable persona.
3. Impact follows when the victim is manipulated into transferring funds, often through repeated payments, withdrawals, or mule-facilitated movement.

Breaches seen in the wild

• Zacks Investment Research breach — Zacks breach exposed 12M customer records including credentials.
• DeepSeek breach — DeepSeek breach exposed 1M+ log lines and sensitive secret keys.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Trust-based fraud is now an identity problem, not only a fraud problem. Romance and investment scams succeed because the attacker can manufacture legitimacy long enough to pass as a trusted counterpart. That puts identity assurance, account provenance, and behavioural verification at the centre of fraud defence. When legitimacy is the product being counterfeited, security teams need to treat identity signals as the first fraud control, not a back-end support function.

Cross-platform scam journeys expose the weakest link in modern trust architecture. The scam often begins in one environment and completes in another, which means no single platform sees the whole attack. This is a governance gap in shared accountability, not just a tooling gap. The practical conclusion is that identity, telco, messaging, and financial services controls need shared abuse signals or the attacker will always choose the least visible hop.

GenAI creates a synthetic confidence layer that outpaces human review. The article’s central warning is that the scammer can now generate fluent language, believable video, and context-aware dialogue that erodes the value of manual inspection. That does not eliminate the need for human intervention, but it does change what humans can realistically adjudicate. Practitioners should assume that subjective trust cues are no longer reliable by themselves.

Identity blast radius is now distributed across the victim journey. The named concept here is the point at which a scam’s impact depends on multiple downstream identities being fooled in sequence, from the platform account to the bank representative to the mule network. Each handoff expands the blast radius because one successful deception enables the next. For practitioners, the implication is that control design has to shrink the number of identity assumptions each stage is allowed to make.

From our research:

• 72% of organisations have experienced or suspect they have experienced a breach of non-human identities, according to The 2024 ESG Report: Managing Non-Human Identities.
• Two-thirds of enterprises have endured a successful cyberattack resulting from compromised non-human identities, with a quarter encountering multiple attacks.
• For a broader governance lens, see Ultimate Guide to NHIs , Key Research and Survey Results for survey data that frames where identity programmes most often lose visibility.

What this signals

Trust-deception campaigns are converging with identity governance weaknesses. The practical signal for practitioners is that verification, monitoring, and escalation need to work as one control plane across customer-facing channels and payment workflows. If those layers remain fragmented, the scammer will keep using the weakest handoff to extend the attack.

85% of organisations lack full visibility into third-party vendors connected via OAuth apps. That same visibility gap is a reminder that abuse often hides in delegated trust relationships, not only in obvious user interfaces, according to The State of Non-Human Identity Security. Identity teams should expect fraud and abuse to move through indirect relationships whenever direct controls tighten.

Financial services and platform teams should prepare for stronger regulatory expectations on scam detection and account integrity. The article points to a future where telcos, messaging platforms, and banks are measured on their ability to identify fraudulent interactions, not just react to them. That shifts identity work from back-office administration to frontline risk containment.

For practitioners

• Instrument early-channel abuse detection Monitor SMS, social, and dating entry points for repeated trust-building patterns, suspicious account creation, and rapid migration to private messaging apps. Pair content analysis with behavioural signals so the same actor can be tracked across channels.
• Add friction to high-risk money movement Use step-up review, transaction holds, and callback verification when a customer requests unusual transfers, withdrawals, or account changes. Train frontline staff to challenge urgency, secrecy, and coaching by a third party.
• Build shared scam intelligence across platforms Create common indicators for fraudulent handles, mule accounts, suspicious URLs, and repeated contact sequences so telco, messaging, and financial services teams can act on the same abuse pattern.
• Target account creation and verification controls Apply stronger checks to new advertisers, new dating profiles, and suspicious business accounts so fraudsters cannot re-establish trust under fresh identities after takedowns.

Key takeaways

• Romance and investment scams are now industrialised trust attacks that use GenAI, platform hopping, and payment manipulation to bypass ordinary vigilance.
• The article’s scale signals a governance failure across contact, verification, and transaction controls, not just a rise in individual victimisation.
• Practitioners should prioritise earlier detection, stronger account verification, and tighter money-movement friction before the scam reaches settlement.

Key terms

• Synthetic trust: Synthetic trust is the appearance of credibility created by manipulation rather than verified identity. In this article’s context, GenAI, impersonation, and repeated contact patterns are used to simulate familiarity long enough to move a victim toward payment or disclosure.
• Trust migration: Trust migration is the movement of a scam from one channel to another, usually from public or monitored contact into private messaging or payment systems. The shift matters because it transfers the victim from one control environment to another while preserving the relationship the attacker built.
• Money mule: A money mule is an account holder or intermediary used to receive and move fraudulent funds on behalf of a criminal. Mule activity is part of the execution layer of many scams because it helps obscure the final destination and complicates recovery, tracing, and reversal.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.

This post draws on content published by Arkose Labs: what a coordinated response to romance and investment scams should look like. Read the original.