TL;DR: Generative AI has made phishing, vishing and impersonation attacks faster, more personalised and harder to spot, according to Ninjio, while shadow AI adds a second layer of employee-driven risk through unapproved tools and sensitive data sharing. Static awareness training is no longer enough; organisations need policy, detection and continuous simulation to reduce human cyber risk.
At a glance
What this is: NHI Mgmt Group's analysis, drawing on Ninjio's blog post: Generative AI is enlarging the human attack surface by scaling phishing, vishing, impersonation and shadow AI risks faster than conventional awareness programmes can absorb.
Why it matters: IAM and security teams need to treat employee behaviour, approved AI use and social-engineering resilience as one governance problem because AI lowers attacker effort while increasing the volume of human mistakes.
Context
Generative AI has changed the cost and pace of social engineering, but the underlying control problem is still trust under pressure. When employees are asked to decide quickly across email, chat, phone and AI-assisted workflows, the security model fails if it assumes people can reliably spot manipulation after the fact.
The governance gap is broader than phishing awareness. Organisations now have to manage approved AI use, the handling of sensitive data inside external tools, and the human tendency to treat fluent machine-generated content as trustworthy. That makes employee risk a combined identity, behavioural and data-handling issue, not just a training topic.
Key questions
Q: Why do AI-generated phishing attacks defeat traditional awareness training?
A: They remove the visual clues that old training relied on, such as spelling errors, odd domains, and obviously fake urgency. When a message mirrors a real role, vendor, or workflow, employees are less likely to question it. That makes context, not just content, the deciding factor in whether the attack succeeds.
Q: Why does shadow AI create a governance risk even when employees mean well?
A: Because intent does not remove exposure. Employees may adopt AI to work faster, but unapproved tools can still receive proprietary content, bypass review expectations, and create compliance gaps. The risk comes from unmanaged data flow and lack of oversight, not from employee malice.
Q: How does vishing differ from email phishing in practice?
A: Vishing uses voice calls, often with caller ID spoofing or social engineering, while email phishing uses fraudulent messages sent through email. In practice, vishing more often pressures a person into bypassing verification steps, resetting credentials, or approving payment or access changes verbally, so call-back controls, passphrases, and out-of-band confirmation reduce risk.
Q: Can deepfake voice scams defeat standard approval workflows?
A: Yes. A convincing deepfake voice can imitate an authorised manager or executive and persuade staff to approve payments, password resets, or access changes if the workflow relies only on voice recognition or urgency. Independent verification through a separate channel, such as a call-back to a known number or two-person approval, is the control that breaks the scam.
Technical breakdown
How Generative AI scales social engineering
Generative AI does not invent a new attack class. It compresses the time needed to research a target, write convincing content and tailor the message to the victim’s role or context. That lowers the cost of phishing, vishing and executive impersonation while increasing the number of attempts an attacker can run in parallel. The technical shift is in content generation and orchestration, not in the basic abuse of trust. Multi-channel campaigns also let attackers move from email to voice or collaboration tools when one channel fails. See the Social Engineering Guide.
Practical implication: Tune detection and training around multi-channel manipulation, not just malformed emails.
Why shadow AI becomes a governance problem
Shadow AI appears when employees use unapproved AI tools outside governed workflows, often to save time rather than to bypass controls. The risk is not only policy non-compliance. It is the loss of visibility into what data is entered, where it is stored, and how the provider may reuse or expose it. In practice, shadow AI creates an uncontrolled shadow workflow for business information, which can intersect with identity governance, privacy obligations and third-party risk management. See the AI Agent Identity Security Buyer's Guide.
Practical implication: Inventory unsanctioned AI use and classify which data types are prohibited in external prompts.
Why voice phishing is harder to block than email-only attacks
Voice phishing adds live conversation, urgency and authority cues that email filters cannot inspect. AI-generated voices and scripted conversational flows can mimic executives, vendors or service desks closely enough to pressure a target into acting before they verify. Because the channel is interactive, the attacker can adapt in real time to the victim’s hesitations or objections. That makes vishing a control challenge for verification processes, not just a content-filtering problem. See the Deepfakes, Social Engineering and AI Impersonation Guide.
Practical implication: Build out-of-band verification into payment, credential reset and access approval workflows.
Threat narrative
Attacker objective: The attacker wants to manipulate employees into authorising access, revealing secrets or disclosing sensitive information at scale.
- Entry begins with AI-generated phishing, vishing or collaboration-platform outreach that is tailored to the target’s role and context.
- Credential or trust abuse follows when the victim shares information, approves a request or accepts the impersonation as legitimate.
- Escalation occurs as the attacker chains email, voice and chat to sustain pressure and obtain more sensitive access or approvals.
- Impact is achieved through stolen credentials, fraudulent payments, exposed data or broader compromise of employee-facing workflows.
Breaches seen in the wild
- MGM Resorts breach 2023: A help desk call gave attackers Okta and Azure admin access at MGM, leading to ransomware, ten days of outages and a $100 million hit.
- Caesars Entertainment breach 2023: Social engineering of an IT support vendor let attackers copy Caesars loyalty database; about $15 million was reportedly paid.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
Generative AI has turned human risk into a throughput problem. The limiting factor in social engineering is no longer just attacker creativity. It is the speed at which personalised, convincing content can be produced and delivered across email, voice and collaboration channels. That changes the economics of fraud and intrusion, because the defender now has to assume a much larger volume of plausible manipulation attempts. Security leaders should therefore measure human risk as a live operational exposure, not a periodic training outcome.
Shadow AI is an access-governance problem, not a productivity side note. When employees enter sensitive information into unapproved AI tools, the organisation loses control over where business data goes and who can reuse it. That creates a governance gap that sits between identity policy, data handling and third-party assurance, and it requires explicit rules for approved use rather than blanket prohibition. A useful framing is AI agent identity security for understanding how governed use and uncontrolled use diverge.
Human trust manipulation: the practical failure mode here is not only weaker awareness, but overreliance on recognition after the fact. AI removes the obvious cues employees were trained to spot, so the organisation must move verification earlier in the workflow and make suspicious requests harder to satisfy. That is especially true where finance, help desk and executive impersonation overlap. Practitioners should treat trust validation as a workflow control, not a training slogan.
The right response is layered human defence, not a single control. Policy sets the boundaries for acceptable AI use, detection reduces exposure to AI-enabled impersonation and training builds the habit of pausing before acting. None of those layers works well alone because the threat spans behaviour, content and process. Practitioners should align awareness, verification and acceptable-use rules so that human resilience is reinforced at every step where trust can be abused.
From our research library:
- 74% of all breaches included the human element, through error, privilege misuse, stolen credentials or social engineering, according to Verizon's 2023 Data Breach Investigations Report.
What this signals
Human risk now behaves like an attack surface. As AI tools become embedded in daily work, the boundary between productivity software and manipulation channel keeps shrinking. That means employee security programmes need to track where trust is being created, not just where it is being broken.
Security teams should expect multi-channel social engineering to become the default pattern, with email, voice and chat working together rather than in isolation. The practical shift is toward governance that combines acceptable use, verification and behaviour shaping so that employees can recognise pressure before they commit an action.
For practitioners
- Implement explicit AI acceptable-use rules Define which AI tools are approved, which data types must never be entered and what approval is required before staff use external AI services for work.
- Add verification to high-risk requests Require out-of-band confirmation for payment changes, credential resets, executive requests and any urgent action that arrives by voice or chat.
- Train employees on AI-enabled social engineering Use simulations that combine email, voice and collaboration-platform lures so staff practise recognising urgency, authority and impersonation across channels.
- Detect shadow AI exposure Track where employees are using unapproved AI tools and classify the data they are sending so privacy and governance teams can intervene early.
Key takeaways
- Generative AI makes familiar social engineering tactics easier to scale, personalise and deliver across channels, which increases the number of credible requests employees must judge quickly.
- Shadow AI expands the same risk into governance and privacy by pushing sensitive work into unapproved tools that security teams cannot see or control well.
- The strongest response combines clear AI policy, verification for high-risk actions, behaviour-focused training and simulations that reflect how attackers now move between email, voice and chat.
Related NHI Mgmt Group resources: Social Engineering Guide · Deepfakes, Social Engineering and AI Impersonation Guide · AI Agent Identity Security Buyer's Guide · Security Awareness and Human Risk Management Guide
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK addresses the attack surface, NIST CSF 2.0 sets the technical controls, and SOC 2 (AICPA) defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| MITRE ATT&CK | TA0001; TA0006; TA0008 — Initial Access; Credential Access; Lateral Movement | AI-enabled phishing and vishing map to the social-engineering and credential-theft stages of intrusion. |
| Recommendation — Map AI-assisted social engineering to TA0001, TA0006 and TA0008 and tune detections for multi-channel lure chains. | ||
| NIST CSF 2.0 | PR.AT-01 — Security Awareness and Skills Training | Employee manipulation and AI-enabled deception are directly addressed through awareness and training outcomes. |
| PR.AA-05 — Access Permissions, Entitlements and Authorizations | High-risk approvals and credential resets depend on reliable authorisation checks before action is taken. | |
| PR.DS-01 — Data-at-Rest is Protected | Shadow AI creates a data-handling problem when employees place sensitive information into external tools. | |
| Recommendation — Build training that teaches employees how AI changes trust cues and verification behaviour. Add stronger approval and verification gates around sensitive requests and access changes. Classify the data employees may not enter into unapproved AI services. | ||
| SOC 2 (AICPA) | CC6.1 — Logical and Physical Access Controls | Shadow AI and impersonation risk affect how organisations control and authorise access to information. |
| Recommendation — Review access pathways to sensitive information and restrict unapproved AI-mediated sharing. | ||
Key terms
- Generative AI: AI designed to create text, code, images, or other content in response to a prompt. It is usually reactive rather than autonomous, which means the main security concern is output quality, leakage, and misuse of generated content rather than independent action.
- Social Engineering: Social engineering is the use of deception, urgency, and authority to persuade a person to reveal information or take a risky action. It targets human decision-making rather than software defects, and often turns legitimate identity workflows into the attack path.
- Phishing: Phishing is a deceptive message or website designed to trick a person into revealing credentials or other sensitive information. In identity terms, it is an unauthorised collection method that turns human trust into downstream account access and potential privilege abuse.
- Vishing: Voice phishing is a social engineering technique that uses phone calls or voice channels to persuade a target to reveal information or approve access. It succeeds by exploiting trust, urgency, and procedural shortcuts, often bypassing technical controls that would have stopped a direct login attack.
Deepen your knowledge
The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, IAM and secrets management in a format built for security and identity practitioners. It helps teams connect identity controls to the wider risk patterns that modern programmes depend on.
Published by the NHIMG editorial team on November 9, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org