AI-generated dating content is eroding trust and verification

At a glance

What this is: This survey shows that AI is reshaping online dating faster than trust controls can keep up, with users reporting more fraud, more synthetic content, and less confidence in what they see.

Why it matters: It matters to IAM practitioners because the same verification, proofing, and trust-assurance gaps appearing in consumer dating platforms are also emerging across NHI, autonomous, and human identity journeys.

By the numbers:

• 84% of UK daters say AI has made online dating harder and less trustworthy.
• More than a quarter of users (28%) say they are not confident in their ability to spot deepfakes or AI-manipulated profiles.
• Over one third (36%) have used an AI companion as an alternative to dating apps.

👉 Read Sumsub's survey findings on AI, trust, and dating app identity risk

Context

AI-generated profiles, messages, and images are weakening trust in online dating because users can no longer assume that identity signals reflect real people. In identity terms, the problem is not just fraud volume, but the collapse of visible cues that proofing and verification workflows usually rely on.

Sumsub’s data suggests a broader trust transition: users are adopting AI to shape their own presentation while also becoming more sceptical of everyone else’s authenticity. That puts pressure on any programme that still assumes human judgement can reliably separate genuine identity from synthetic content without stronger verification controls.

For IAM teams, the lesson is that once a channel becomes saturated with machine-generated content, the burden shifts from content moderation to assurance design. That same shift now shows up in NHI governance and autonomous systems, where trust cannot rest on appearance alone.

Key questions

Q: How should security teams verify identity when AI-generated profiles look authentic?

A: Security teams should stop relying on visual realism or message quality as proof of identity. Stronger verification should combine liveness checks, device and account signals, behavioural risk scoring, and step-up review at moments where trust is being extended. The goal is to verify the claimant, not the quality of the synthetic content.

Q: Why do AI-generated messages and images weaken trust in digital identity flows?

A: They weaken trust because they remove the reliability of cues people traditionally use to judge authenticity. When anyone can generate convincing text or images at low cost, appearance stops being a dependable proxy for real identity. That means assurance must move from subjective judgement to explicit verification and provenance checks.

Q: What breaks when identity checks depend on human judgement in AI-heavy channels?

A: Human judgement becomes inconsistent once synthetic content is personalised and widely available. Reviewers can miss subtle manipulation, and users can be trained to distrust legitimate interactions. The result is both more fraud exposure and more false suspicion, which reduces platform confidence unless controls are automated and evidence-based.

Q: Who is accountable when AI-generated identity deception succeeds on a platform?

A: Accountability usually sits with the platform that set the trust model, the verification process, and the abuse response path. If a service allows synthetic identity signals to move users into high-trust interactions without adequate proofing, responsibility is not shifted to the victim. Governance has to cover both design and enforcement.

Technical breakdown

AI-generated dating profiles and the collapse of identity cues

Dating platforms normally depend on weak but familiar trust signals such as profile photos, short bios, and conversational style. AI removes the reliability of those cues by making them cheap to fabricate at scale. Deepfake images, polished messages, and synthetic companionship can all look authentic enough to pass casual inspection, which means identity assurance must move earlier in the flow and rely on stronger verification than user intuition.

Practical implication: teams should treat visible profile signals as untrusted input and design verification steps that do not depend on human pattern recognition alone.

AI fraud, deepfakes, and the limits of manual review

Manual review works poorly when fraud artefacts are individually plausible but collectively inconsistent. AI-generated content can be tailored to each target, which reduces the value of static rules and makes it harder for reviewers to separate legitimate use of AI from deceptive impersonation. The core problem is that scale and personalisation now sit on the attacker’s side, while review teams are still trying to adjudicate authenticity one case at a time.

Practical implication: organisations need risk scoring, provenance signals, and step-up checks that trigger before trust is granted, not after an abuse report arrives.

Verification in AI-heavy user journeys

When AI becomes part of the user journey itself, verification has to distinguish between acceptable augmentation and deceptive identity substitution. That requires a layered model that can detect suspicious behavioural patterns, surface manipulated media, and tie actions back to a verified account lifecycle. In practice, the trust boundary is no longer the login screen, it is every point where identity claims can be generated, edited, or amplified by AI.

Practical implication: security teams should map where identity claims are created, transformed, and consumed, then place controls at each transition point.

Breaches seen in the wild

• Cisco DevHub NHI breach — IntelBroker exploited exposed Cisco credentials, API tokens and keys in DevHub.
• DeepSeek breach — DeepSeek breach exposed 1M+ log lines and sensitive secret keys.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Synthetic identity is now a governance problem, not just a fraud problem. The article shows that users are not merely encountering more scams, they are adapting to an environment where authenticity itself is being industrialised. That changes the governance question from “how do we block bad actors” to “how do we prove identity when the medium is synthetic by default.” Practitioners should treat this as an assurance design issue across digital channels.

Weak trust signals fail once AI can manufacture them at will. Photo quality, message fluency, and conversational consistency used to support informal confidence in identity claims. AI now reproduces those signals cheaply, which means controls built around human judgement lose effectiveness as attack quality rises. The implication is that programmes need provenance, verification, and behavioural assurance rather than reliance on appearance.

Identity proofing assumptions break when users also become AI users. This survey shows people are simultaneously using AI to craft profiles and becoming less able to trust the output they receive. That creates a feedback loop where the platform must distinguish legitimate augmentation from deceptive impersonation at the same time. The implication is that identity governance has to account for AI-mediated self-presentation, not just malicious abuse.

Consumer trust erosion is an early warning for enterprise identity assurance. The same pattern appears whenever a channel allows synthetic content to outpace verification. Dating is simply an exposed test case for a broader shift in digital identity governance, where proofing logic must adapt to AI-generated claims across onboarding, engagement, and support. Practitioners should expect similar pressure in customer identity and NHI-adjacent workflows.

From our research:

• The average estimated time to remediate a leaked secret is 27 days, despite 75% of organisations expressing strong confidence in their secrets management capabilities, according to The State of Secrets in AppSec.
• Organisations maintain an average of 6 distinct secrets manager instances, creating fragmentation that undermines centralised control, according to GitGuardian and CyberArk.
• For a broader identity baseline, see Ultimate Guide to NHIs for how governance, lifecycle, and visibility controls fit together.

What this signals

Synthetic identity pressure is now part of the wider trust-assurance problem. With 43% of security professionals already concerned about AI systems learning and reproducing sensitive information patterns from codebases, the challenge is no longer isolated to consumer fraud. Teams should expect the same trust erosion to appear wherever identity claims, secrets, or content can be reproduced by machines before governance catches up.

The practical signal for practitioners is that verification must be treated as a workflow property, not a front-end feature. If identity claims can be created, edited, or amplified by AI, then moderation, proofing, and abuse response all need to be aligned to the same trust model.

For teams building stronger assurance programmes, the relevant comparator is not user experience polish but control placement. The more AI can generate convincing output, the more identity governance has to depend on evidence, provenance, and step-up validation at the point of trust extension.

For practitioners

• Separate authenticity checks from content quality signals Do not treat polished text, strong grammar, or realistic images as evidence of a real person. Build verification paths that use device, document, behavioural, or liveness signals before trust is extended.
• Add step-up verification at high-trust moments Trigger stronger checks when users exchange contact details, request payment, or move off-platform. The control should activate at the point where fraud risk changes, not only at sign-up.
• Track synthetic-content exposure as an assurance metric Measure how often users encounter AI-generated profiles, messages, or images and correlate that with reporting, abandonment, and verification failure rates.
• Review trust design for AI-mediated identity journeys Map where users can create, edit, or amplify identity claims with AI tools, then align moderation, proofing, and review controls to those points in the journey.

Key takeaways

• AI-generated content is eroding authenticity signals that consumer identity systems have historically depended on.
• The scale problem is not just more fraud, but more plausible deception that users struggle to detect.
• Practitioners should move verification earlier and anchor trust in evidence, provenance, and step-up controls.

Key terms

• Synthetic Identity: An identity presentation created or heavily altered by AI or other tooling so that it appears real to a reviewer or platform. In practice, synthetic identity can include generated photos, rewritten bios, or polished messages that imitate legitimate users and weaken trust signals.
• Identity Proofing: The process of establishing that a claimed identity is real enough to be trusted for a given interaction. In AI-heavy channels, proofing has to rely on evidence, liveness, and verification signals rather than visual realism or conversational fluency.
• Step-Up Verification: An additional verification check triggered when risk increases during a user journey. It is used when a normal sign-in or profile check is no longer sufficient, such as before sensitive contact exchange, payment, or account escalation.

Deepen your knowledge

AI-driven trust erosion and verification design are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are adapting identity assurance for AI-heavy journeys, it is worth exploring.

This post draws on content published by Sumsub: AI-generated content is making online dating harder and less trustworthy. Read the original.