AI-driven fraud is exploiting trust gaps in identity and data sharing

At a glance

What this is: This is a SumSub podcast conversation about how AI, deepfakes, and information-sharing gaps are making fraud more targeted and more successful.

Why it matters: It matters to IAM practitioners because the same trust failures that help fraudsters also weaken human verification, exception handling, and identity assurance across customer, employee, and non-human access paths.

👉 Read SumSub's conversation on AI-driven fraud, deepfakes, and trust gaps

Context

Fraud now succeeds less because targets are careless and more because attackers can manufacture convincing proof, timing, and context faster than verification workflows can respond. That shifts the problem from awareness alone to assurance, where identity, device, and behavioural signals have to withstand synthetic manipulation.

For IAM and fraud teams, the issue is not only account takeover. It is also the collapse of the trust assumptions behind step-up checks, call-centre verification, recovery processes, and exception handling when deepfakes or AI-generated narratives can mimic legitimacy. The article frames that shift through a fraud practitioner’s lens rather than a pure technical one.

Key questions

Q: How should security teams reduce fraud risk in account recovery workflows?

A: Security teams should require multiple independent proofs for recovery actions, especially when the action can move money, change credentials, or restore access. Voice, video, and challenge questions should be treated as weak signals, not final authority. Stronger workflows combine step-up checks, transaction context, and manual review for high-risk cases.

Q: Why do AI deepfakes make identity verification less reliable?

A: AI deepfakes make verification less reliable because they can imitate a person well enough to satisfy a single channel of proof. That turns many checks into tests of media realism instead of identity. Organisations need layered verification that does not depend on one voice call, one video call, or one scripted question.

Q: What do fraud teams get wrong about shared threat intelligence?

A: They often treat intelligence sharing as a back-office task rather than a live control. When scam patterns are not shared quickly across teams or organisations, attackers can reuse the same narrative for months. Faster escalation, shared indicators, and linked case handling shorten the fraud lifecycle.

Q: What should organisations do about quantum risk in identity and secrets management?

A: Organisations should inventory secrets and cryptographic dependencies that must remain trustworthy for years, then prioritise the systems with the longest confidentiality horizon. The goal is not panic planning. It is reducing exposure where today’s encryption assumptions would create future identity and data loss if they weaken.

Technical breakdown

AI-generated fraud narratives and synthetic identity proof

AI lowers the cost of producing plausible fraud at scale. Attackers can combine personal data, stolen account history, and synthetic voice or text to create a believable identity story that fits a victim, a help desk, or a payment workflow. The mechanism is not just impersonation. It is context fabrication, where the fraudster supplies enough coherent detail to trigger trust in people and systems that were built to verify isolated facts rather than narrative consistency.

Practical implication: treat narrative coherence as a risk signal and require multiple independent verification steps for sensitive actions.

Deepfakes and the failure of single-channel verification

Deepfakes weaken controls that rely on one channel of proof, such as voice callbacks, video calls, or scripted security questions. When synthetic audio or video can closely match a real person, the control becomes a test of media quality rather than identity assurance. The real weakness is that many verification processes assume authenticity can be established from a single interaction, when fraud now succeeds by controlling the interaction context.

Practical implication: remove single-channel approval paths for recovery, payout, and privileged access decisions.

Collaboration gaps that let fraud persist

Fraud campaigns often endure because organisations do not share enough data quickly enough to expose patterns across victims. A scam can keep working for months if each target sees only a small piece of the abuse and the wider network signal never reaches the right defenders. This is a governance problem as much as a detection problem, because fragmented reporting gives attackers time to scale before the pattern becomes obvious.

Practical implication: build cross-team escalation and intelligence-sharing workflows so repeated fraud patterns are visible before they mature.

Breaches seen in the wild

• Cisco DevHub NHI breach — IntelBroker exploited exposed Cisco credentials, API tokens and keys in DevHub.
• DeepSeek breach — DeepSeek breach exposed 1M+ log lines and sensitive secret keys.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Fraud has become an identity assurance problem, not just a user-awareness problem. The episode shows how modern scams exploit the gap between what a person can convincingly present and what a verifier can reliably prove. Once AI can generate believable context on demand, the control question shifts from 'is the user suspicious?' to 'can the workflow still establish trust under synthetic pressure?' Practitioners should treat fraud operations and IAM as overlapping assurance disciplines.

Single-channel verification is now a brittle assumption. Voice, video, and scripted challenge questions all fail when the attacker can synthesize one channel well enough to satisfy the process. That is not a tooling gap alone. It is a governance failure in which the organisation assumes one proof signal can carry identity assurance across high-risk actions. Practitioners should re-evaluate which journeys still rely on one channel as the final decision point.

Collaboration latency is a control failure, not an administrative inconvenience. The fake Duke of Marlborough example in the article worked for months because hotels would not share data with competitors. That kind of delay creates a fraud persistence window that attackers can exploit repeatedly across victims and sectors. In identity terms, the control failure is fragmented visibility. Practitioners should view information-sharing delays as an active part of the attack surface.

Trust infrastructure is now part of the fraud kill chain. The article makes clear that attackers succeed by combining synthetic identity cues with gaps in organisational coordination. That means prevention is no longer only about detecting bad actors after the fact. It also requires reducing the number of places where a convincing but unverified narrative can trigger access, payment, or recovery decisions. Practitioners should map every high-risk journey for trust injection points.

Quantum risk is a forward signal, not a distant abstraction. The discussion of quantum computing matters because identity and fraud programmes still depend on cryptographic assumptions that many business processes treat as fixed. Even before quantum-era compromise arrives, the possibility changes how teams should think about long-lived credentials, archived secrets, and the durability of evidence chains. Practitioners should treat cryptographic longevity as a governance variable, not a background detail.

From our research:

• 79% of organisations have experienced secrets leaks, with 77% of these incidents resulting in tangible damage, according to the Ultimate Guide to NHIs.
• Only 5.7% of organisations have full visibility into their service accounts, which shows how often defenders are operating without a complete identity picture.
• For a deeper operational lens, the Guide to the Secret Sprawl Challenge shows how exposed credentials and fragmented secret storage expand fraud-adjacent risk.

What this signals

Trust verification is becoming a shared IAM and fraud-control concern. As AI makes impersonation more convincing, identity programmes will need to measure assurance quality, not just authentication success. The teams that can connect recovery, help desk, and payment controls will be better placed to catch synthetic abuse before it becomes loss.

Identity programmes should expect more demand for joined-up evidence. Fraudsters benefit when signals sit in separate silos, while defenders need a single picture of repeated narratives, reused artefacts, and suspicious transaction context. That makes case linkage and escalation design a governance issue, not just an investigation task.

For practitioners

• Harden high-risk recovery journeys Require at least two independent verification methods before password resets, payout changes, or account recovery actions are approved. Do not let voice, video, or security questions stand alone when the transaction has high fraud value.
• Add fraud intelligence to identity workflows Feed confirmed scam patterns into help desk, payment, and access workflows so repeated narratives and reused indicators can be flagged quickly. Cross-team visibility matters more when fraud is coordinated across many victims.
• Review legacy trust assumptions Audit every workflow that still assumes a real person will always present consistent proof in real time. If a process depends on a single conversation or one challenge question, it is already overexposed.
• Prepare for cryptographic transition risk Inventory long-lived secrets, certificates, and archived sensitive data that would remain valuable if encryption strength erodes over time. Focus first on systems where confidentiality needs to survive for years, not months.

Key takeaways

• AI-assisted fraud is shifting the problem from obvious deception to high-confidence synthetic legitimacy.
• Where verification depends on one channel or one conversation, fraudsters now have enough realism to pass.
• Organisations need stronger cross-team signal sharing and multi-factor proof for high-risk identity actions.

Key terms

• Synthetic identity proof: A fabricated or manipulated set of signals used to persuade a verifier that a person or account is legitimate. In fraud operations, synthetic proof can combine real data, AI-generated media, and contextual detail to bypass checks that only validate one signal at a time.
• Assurance signal: Any observable fact used to decide whether an identity claim should be trusted, such as device posture, voice, transaction context, or behavioural history. The strength of an assurance signal depends on how hard it is to fake and how independently it can be verified.
• Collaboration latency: The delay between one team detecting a fraud pattern and another team being able to act on it. In practice, this gap gives attackers more time to reuse the same narrative or method across victims, turning slow information sharing into a measurable control weakness.

Deepen your knowledge

AI-driven fraud and identity assurance are covered in the NHI Foundation Level course, the industry's only accredited NHI security programme. If you are tightening recovery, verification, and secrets governance in response to synthetic abuse, it is worth exploring.

This post draws on content published by SumSub: an episode on AI-driven fraud, deepfakes, and collaboration gaps. Read the original.