Topic starter
02/08/2026 12:29 pm
TL;DR: AI-generated deepfakes and fabricated content are making social engineering and fraud more persuasive by exploiting emotion, bias, and misplaced trust in what people see and hear, according to Anomali. The control problem is shifting from spotting fakes to building resilience against manipulation before users act on it.
Editorial analysis by NHI Mgmt Group, based on content published by Anomali: “Spotting AI-Generated Disinformation and Deepfakes Online”.
By the numbers:
- One analysis reported a tenfold rise in deepfake related fraud between 2022 and 2023, as cited by Anomali.
Key questions
Q: What breaks when people rely on deepfake detection as the main control?
A: Human spotting fails because even careful observers can miss convincing synthetic audio, video, or text.
Q: Why do deepfakes make social engineering more effective?
A: Deepfakes make social engineering more effective because they add sensory credibility to the usual pressure tactics.
Q: What are the signs that a disinformation or deepfake attack is succeeding?
A: Common signs include unusual urgency, pressure to bypass normal process, emotionally charged language, unexpected channel changes, and requests that depend on immediate trust in a familiar person.
Practitioner guidance
- Implement out-of-band verification for high-risk requests Require callback or secondary-channel confirmation for payments, account changes, and urgent executive instructions, especially when the request is unusual or time-sensitive.
- Train staff on influence tactics, not just fake-media signs Teach teams to recognise urgency, authority, social proof, reciprocity, and confirmation bias so they can slow down before acting on suspicious content.
- Add provenance checks to executive and finance workflows Use approved contact directories, signing, and change-control rules so that a convincing voice or video cannot authorise material action by itself.
Bottom line: AI-generated disinformation increases social engineering risk because it turns familiar-looking media into a vehicle for urgency, authority, and emotional pressure.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
02/08/2026 1:27 pm
Deepfake disinformation is now an identity risk, not just a media-verification problem. When a forged face or voice can stand in for a trusted executive, the control question shifts to how organisations confirm who is really making a request. That makes identity verification, out-of-band challenge, and approval hygiene part of the defence, not just communications policy. The practical conclusion is that identity assurance now has to extend into everyday human decision points.
A few things that frame the scale:
- Deepfake scams now account for nearly $12 billion in fraud losses globally.
A question worth separating out:
Q: How should organisations govern high-risk requests when synthetic media is possible?
A: They should require independent verification for payments, account changes, and sensitive approvals, and they should document who can approve what under what conditions. The goal is to make a convincing voice, face, or message insufficient on its own to trigger irreversible action.
👉 Read our full editorial: AI-generated disinformation is reshaping social engineering risk