They increase risk because attackers can tailor messages, voices, and context to look legitimate at scale. That makes users more likely to approve requests, reveal credentials, or sign malicious actions, especially when the organisation relies on human judgment instead of strong verification for privileged workflows.
Why This Matters for Security Teams
AI-powered phishing and deepfakes change credential risk because they reduce the friction attackers traditionally faced when impersonating trusted people or systems. With convincing language, cloned voices, and realistic context, an attacker can bypass habits that once acted as informal checks, such as spotting bad grammar or recognising an unfamiliar tone. That matters most when a request triggers password resets, MFA approvals, or access to privileged systems.
The practical issue is not only deception at first contact. It is the way a persuasive message can move a target into revealing secrets, approving a login, or authorising a workflow that should have required stronger verification. Security teams should treat this as an identity assurance problem, not only a user awareness problem, and anchor controls in guidance such as the NIST SP 800-63 Digital Identity Guidelines and the NIST Cybersecurity Framework 2.0.
In practice, many security teams encounter this only after a convincing impersonation has already triggered a credential reset, payment request, or privileged approval.
How It Works in Practice
AI makes phishing more effective by improving personalisation, timing, and delivery quality. Rather than sending a generic lure, attackers can use publicly available information, breached data, or prior correspondence to build messages that mirror real business processes. Deepfakes extend that advantage by adding synthetic voice or video, which can make urgent requests seem to come from a known executive, supplier, or help desk agent.
From a credential-risk perspective, the attacker is usually trying to reach one of three outcomes: collect a password or token, coerce an MFA approval, or redirect a legitimate authentication flow. In mature environments, the best defence is layered control design, not trust in human recognition alone. That means tightening identity proofing, hardening resets, limiting standing privilege, and requiring stronger verification for sensitive actions. The control intent aligns well with NIST SP 800-53 Rev 5 Security and Privacy Controls, especially for authentication, access enforcement, and verification processes.
Operationally, teams should look for:
- phishing-resistant authentication for privileged and high-risk users
- out-of-band verification for password resets, bank detail changes, and MFA re-enrolment
- controls that prevent approval of requests based only on voice, video, or email appearance
- logging and alerting for unusual authentication, reset, and escalation patterns
- identity governance for service accounts and other non-human identities that can be abused after initial compromise
This is where the overlap with NHI matters: if an attacker can trick a human into approving a workflow, they may inherit access to APIs, automation, or service credentials that were never meant to be exposed through a person. The OWASP Non-Human Identity Top 10 is useful here because it highlights how secrets, tokens, and machine access can become the real prize after a social-engineering step.
These controls tend to break down when help-desk resets, executive approvals, and contractor access are handled through ad hoc exceptions because the attacker only needs one weak process path.
Common Variations and Edge Cases
Tighter verification often increases user friction and support overhead, requiring organisations to balance stronger credential protection against operational speed. That tradeoff is especially visible in executive communications, time-sensitive finance workflows, and incident response, where people are tempted to override normal checks. Current guidance suggests that the answer is not to remove verification, but to define risk-based exceptions with compensating controls and clear approval boundaries.
Deepfake risk also varies by environment. In small organisations, the main exposure may be a single convincing voice call to finance or IT support. In larger enterprises, the attack surface expands across multilingual operations, outsourced help desks, and distributed identity systems. There is no universal standard for deepfake-specific verification yet, so practitioners should apply existing identity assurance principles consistently rather than wait for a novel control category.
For organisations with strong AI usage, the issue can widen further. If employees already rely on chatbots or AI assistants for routine work, attackers may mimic those interactions or impersonate an internal assistant to request credentials or approvals. That is why AI governance and identity governance need to be connected, even when the immediate question is simply about phishing. The common failure point is assuming a trusted tone proves trusted intent.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST CSF 2.0, NIST SP 800-63 and NIST SP 800-53 Rev 5 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | PR.AC-1 | Identity proofing and access control reduce credential abuse from social engineering. |
| NIST SP 800-63 | IAL2 | Stronger identity assurance helps resist impersonation in password reset and recovery flows. |
| NIST SP 800-53 Rev 5 | IA-2 | Multi-factor authentication limits the value of stolen credentials from phishing. |
| OWASP Non-Human Identity Top 10 | NHI-05 | Non-human credentials become a follow-on target after human compromise or approval. |
Enforce MFA for privileged and high-risk access, and prefer phishing-resistant methods where possible.
Related resources from NHI Mgmt Group
- Why do AI-powered IDE extensions increase developer credential risk?
- Why do AI systems increase the risk of credential misuse?
- Why do AI-generated phishing campaigns increase risk for public-sector agencies?
- Why do adversary-in-the-middle phishing kits increase identity risk beyond ordinary credential theft?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on July 14, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
