Traditional controls can slow attackers, but they will not fully contain AI-assisted abuse if they are not adapted. Deepfakes can bypass visual trust, sponge attacks can consume LLM resources, and impersonation can exploit weak callback or approval processes. The consequence is greater exposure to fraud, service disruption, and decision errors unless teams add stronger verification and AI-specific monitoring.
Why Traditional Controls Break First
Traditional controls were designed to verify known users, known systems, and familiar attack paths. Deepfakes and AI-assisted impersonation weaken those assumptions by making voice, video, email, and chat look routine when they are not, while sponge attacks exploit the fact that many AI services still lack strong cost, rate, and workload boundaries. The result is not control failure in one place, but control drift across verification, approval, and service consumption.
Organisations should expect the first failure to appear where trust is operationalised, not where policy is written. A callback process, a helpdesk approval, or a manual review can become the weakest link if the reviewer is forced to trust the channel instead of verifying the requester through an independent step. In practice, many teams discover the gap only after a convincing impersonation has already triggered a payment, reset, or data release.
For a useful baseline on identity and access control expectations, see NIST SP 800-53 Rev 5 Security and Privacy Controls.
How These Attacks Work Against Normal Operations
Deepfakes and impersonation attacks succeed when the defender treats a single signal as sufficient proof. A realistic synthetic voice can pressure a helpdesk, a synthetic face can defeat casual visual checks, and a well-formed message can mimic an executive or vendor with enough confidence to trigger action. The control problem is less about the media itself and more about over-trusting any one communication channel.
Sponge attacks work differently. They do not need to “break” the model in the classic sense. Instead, they try to burn through tokens, memory, context windows, or queue capacity so that legitimate users face delays, degraded quality, or increased cost. If the service lacks quotas, abuse detection, and request shaping, a small number of malicious prompts can create outsized operational impact.
- Deepfake abuse targets human judgement and weak out-of-band verification.
- Sponge attacks target service availability, compute spend, and response quality.
- AI-assisted impersonation often blends social engineering with workflow abuse.
- Manual approval steps fail when they do not require independent confirmation.
The common pattern is that the attacker does not need to win every control, only the one that turns trust into action. These controls tend to break down when verification is treated as a formality rather than a separate decision path.
Common Variations and Edge Cases
Tighter verification often increases friction, so organisations have to balance speed against assurance. That tradeoff is most visible in customer support, finance, executive workflows, and any process where a fast callback or voice approval was previously considered acceptable. The right response is not to remove human judgement, but to make high-impact actions require evidence that a synthetic impersonation cannot easily satisfy.
Some environments will also need different controls for different AI risks. A deepfake call may call for stronger identity proofing and second-channel confirmation, while a sponge attack may call for throttling, cost controls, circuit breakers, and observability on abnormal usage patterns. Treating all AI abuse as the same problem usually leads to overbuilt approval gates on one side and weak service protection on the other.
There is no universal standard for this yet, but current guidance suggests that organisations should separate “who seems real” from “what action is allowed” and should not let a realistic voice or face substitute for policy-based approval. That distinction matters most where a single mistaken approval can trigger material loss or operational disruption.
Risk and Threat Considerations
These attacks create both fraud risk and operational resilience risk. Deepfakes and impersonation undermine trust in the channels people use to authorise payments, resets, and sensitive requests, while sponge attacks can turn AI capability into a denial-of-service and cost-exhaustion vector. The exposure is highest where organisations rely on speed, familiarity, or one-step approval instead of independent verification.
Failure mechanism: The attacker exploits a control that assumes the channel itself proves legitimacy, then uses synthetic media or workflow manipulation to obtain action. In sponge attacks, the attacker instead abuses resource consumption limits, causing queue buildup, latency, service degradation, or unexpected spend.
Impact: The result can be fraudulent transactions, unauthorised account changes, misleading decisions, degraded AI service quality, and higher operational cost. In the worst case, teams continue trusting a compromised process long after the first successful impersonation or abuse pattern.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK address the attack and risk surface, while NIST CSF 2.0 and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | PR.AC-1 — Identity Management, Authentication and Access Control | AI impersonation succeeds when access decisions trust weak identity signals. |
| DE.CM-1 — Monitoring for Anomalies and Events | Sponge attacks create abnormal usage patterns and service degradation. | |
| PR.PT-3 — Least Functionality | Resource exhaustion is reduced by limiting exposed AI service functionality. | |
| Recommendation — Require stronger identity verification before approving high-impact requests. Monitor for anomalous AI usage and alert on consumption spikes. Limit exposed AI functions to the minimum needed for business use. | ||
| CIS Controls v8 | 6 — Access Control Management | Impersonation abuse often succeeds through weak approvals and access changes. |
| 8 — Audit Log Management | Synthetic abuse and sponge attacks require reliable detection evidence. | |
| 12 — Network Infrastructure Management | Sponge attacks exploit insufficient throttling and service boundaries. | |
| Recommendation — Tighten approval paths for sensitive access and transaction changes. Log approvals, denials, and abnormal AI usage for investigation. Apply rate limits and isolation boundaries to protect AI services. | ||
| MITRE ATT&CK | T1566 — Phishing | AI-assisted impersonation often uses convincing phishing and social engineering. |
| T1499 — Endpoint Denial of Service | Sponge attacks aim to exhaust AI service resources and disrupt availability. | |
| Recommendation — Hunt for synthetic social-engineering patterns in user-facing channels. Detect and block requests that exhaust model or service capacity. | ||
Practitioner Guidance
What to prioritise: Put independent verification in front of any action that can move money, change access, or release sensitive data. If a request can be completed using only a voice call, video call, or email thread, it is not yet strong enough for high-impact operations.
What to verify: Check whether the control path includes a second channel, a known callback destination, or a separate approver who is not relying on the same synthetic media. For AI services, verify that rate limits, quotas, and abuse detection are actually enforced under load, not only documented.
Decision rule: If the request is urgent and consequential, treat urgency as a risk signal rather than proof. Escalate to stronger confirmation before approving, because urgency is one of the easiest conditions for an impersonator to exploit.
What practitioners underestimate: The most dangerous failure is often the process that works “well enough” in ordinary cases. That is exactly where deepfakes, impersonation, and sponge attacks cause the most damage, because the organisation has already encoded trust in the shortcut.
Practitioner takeaway: The objective is not to distrust every AI interaction, but to ensure that high-consequence decisions can survive synthetic persuasion and resource abuse without depending on human intuition alone.
Related resources from NHI Mgmt Group
- What happens when hotels rely on traditional security controls alone against AI-driven fraud?
- What breaks when organisations rely only on static access controls against AI-driven impersonation?
- What breaks when organisations rely on passwords and weak session controls against AI-assisted account takeover?
- What happens when organisations rely on training alone instead of stronger identity controls against phishing?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 16, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
