Notifications
Clear all
Topic starter
02/08/2026 12:29 pm
TL;DR: AI-generated deepfakes and disinformation are now routinely used to strengthen social engineering, fraud, and influence operations, with studies cited in the source showing a tenfold rise in deepfake-related fraud and a 3,000% increase in attempted deepfake fraud during 2023. The control problem is no longer detection alone, but resilience against emotionally driven manipulation and trust abuse.
NHIMG editorial — based on content published by Anomali: Spotting AI-Generated Disinformation and Deepfakes Online
By the numbers:
- In recent years, the volume of social engineering attacks, scams and fabricated media driven by deepfakes has increased significantly, with one analysis reporting a tenfold rise in deepfake related fraud between 2022 and 2023.
- A CDC backed meta-analysis across 56 studies involving more than 86,000 participants found average human accuracy for identifying deepfakes was only slightly above chance, typically between 57 and 60 percent.
- One early 2024 case used a live deepfake video call to impersonate a company’s CFO and trick an employee into transferring approximately £20 million GBP.
Questions worth separating out
Q: How should organisations handle identity verification when deepfakes can mimic real users?
A: Organisations should stop treating visual similarity as proof of identity and move high-risk workflows toward cryptographic verification, issuer trust, and device-bound proof of possession.
Q: Why do deepfakes make social engineering more effective?
A: Deepfakes make social engineering more effective because they add sensory credibility to the usual pressure tactics.
Q: What do organisations get wrong about spotting AI-generated disinformation?
A: They often assume that awareness alone is enough, or that people can reliably identify fakes by sight, sound, or instinct.
Practitioner guidance
- Implement out-of-band verification for high-risk requests Require a second trusted channel before approving payments, password resets, privileged access, or executive requests that arrive by voice, video, or message.
- Add deepfake scenarios to fraud and security playbooks Test how teams respond when a fake executive call, synthetic customer video, or AI-generated threat message is used to pressure staff into action.
- Standardise content verification checks Create a simple workflow that asks staff to check source reputation, context, corroborating reports, and visual or audio anomalies before sharing or acting on suspicious material.
What's in the full article
Anomali's full post covers the practical detection methods and reference resources this analysis intentionally leaves at a higher level:
- Detailed visual and audio anomaly checks for deepfake images, audio, and video that can be used in incident triage.
- Step-by-step guidance on using reverse image search and metadata review to validate suspicious media before acting.
- A structured verification process for handling suspicious messages, calls, and clips in security and fraud workflows.
- A curated set of external resources for fact checking, forensic review, and media verification.
👉 Read Anomali's analysis of AI-generated disinformation and deepfakes →
AI-generated disinformation and deepfakes: what practitioners need to know?
Explore further
02/08/2026 1:27 pm
AI-generated deception is now a trust-verification problem, not just a content-moderation problem. Deepfakes matter because they exploit the assumptions behind human approval workflows, especially where identity is established through sight or sound. That creates governance exposure in finance, HR, support, and executive operations. Practitioners should treat manipulated media as a control failure in verification design, not merely a user awareness issue.
A question worth separating out:
Q: How can security teams reduce the impact of manipulated media?
A: Security teams can reduce impact by making manipulated media less useful to attackers. That means controlled approval paths, strict callback procedures, strong identity proofing, and incident playbooks that treat suspicious media as a trigger for verification rather than immediate action. The goal is to slow the decision until independent evidence supports it.
👉 Read our full editorial: AI-generated disinformation is reshaping social engineering risk
