Join our Newsletter — 33% off our NHI Course

Is shadow AI making phishing attacks harder to detect?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 2 years ago
Posts: 20217
Topic starter  

TL;DR: Generative AI is making phishing, vishing and impersonation attacks more personalised and faster to produce, Ninjio reports, while shadow AI expands exposure through unapproved tools and data sharing. Human-risk programmes now need policy, detection and repeated simulations that teach people to spot manipulation in real time.

Editorial analysis by NHI Mgmt Group, based on content published by Ninjio: “Human Cybersecurity in the AI Era: What AI Means for Employee Risk and Security Awareness Training”.

Key questions

Q: Why does shadow AI create a governance risk even when employees mean well?

A: Because intent does not remove exposure.

Q: How does vishing differ from email phishing in practice?

A: Vishing uses voice calls, often with caller ID spoofing or social engineering, while email phishing uses fraudulent messages sent through email.

Q: Why do AI-generated phishing attacks defeat traditional awareness training?

A: They remove the visual clues that old training relied on, such as spelling errors, odd domains, and obviously fake urgency.

Practitioner guidance

  • Implement explicit AI acceptable-use rules Define which AI tools are approved, which data types must never be entered and what approval is required before staff use external AI services for work.
  • Add verification to high-risk requests Require out-of-band confirmation for payment changes, credential resets, executive requests and any urgent action that arrives by voice or chat.
  • Train employees on AI-enabled social engineering Use simulations that combine email, voice and collaboration-platform lures so staff practise recognising urgency, authority and impersonation across channels.

Bottom line: Generative AI makes familiar social engineering tactics easier to scale, personalise and deliver across channels, which increases the number of credible requests employees must judge quickly.

Glossary terms: Generative AI, Social Engineering, Phishing, Vishing

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 6 months ago
Posts: 20707
 

Generative AI has turned human risk into a throughput problem. The limiting factor in social engineering is no longer just attacker creativity. It is the speed at which personalised, convincing content can be produced and delivered across email, voice and collaboration channels. That changes the economics of fraud and intrusion, because the defender now has to assume a much larger volume of plausible manipulation attempts. Security leaders should therefore measure human risk as a live operational exposure, not a periodic training outcome.

A few things that frame the scale:

  • 74% of all breaches included the human element, through error, privilege misuse, stolen credentials or social engineering, according to Verizon's 2023 Data Breach Investigations Report.

A question worth separating out:

Q: Can deepfake voice scams defeat standard approval workflows?

A: Yes. A convincing deepfake voice can imitate an authorised manager or executive and persuade staff to approve payments, password resets, or access changes if the workflow relies only on voice recognition or urgency. Independent verification through a separate channel, such as a call-back to a known number or two-person approval, is the control that breaks the scam.

👉 Read our full editorial: Generative AI expands phishing, vishing and shadow AI risks



   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.