AI-powered fraud is outpacing enterprise defensive AI readiness

At a glance

What this is: This research examines how enterprises are responding to AI-powered fraud, with bots now representing a large share of attacks and most organisations still underprepared for adversarial AI.

Why it matters: It matters because IAM, account security, and fraud controls increasingly need to defend both human and non-human access paths against AI-driven abuse.

By the numbers:

• 40% of all attacks on enterprises in the last year were AI-powered bots.
• 88% of enterprises saw an increase in AI-powered bot attacks in the last two years.
• 51% of all respondents don’t have enough talent with both AI and cybersecurity skills.
• 1% of AI-powered bots cracked the AI-resistant challenges, challenges in a recent A/B test.

👉 Read Arkose Labs' research on AI-powered fraud and AI-resistant controls

Context

AI-powered fraud is no longer a niche threat. The core problem is that attackers can iterate faster than many enterprises can approve, deploy, and tune defensive controls, which leaves account security, bot management, and revenue-protecting applications exposed to adversarial AI.

For IAM and fraud teams, this is a governance problem as much as a detection problem. The article points to a gap between the speed of AI-enabled abuse and the slower operating model of enterprise security programmes, especially where sign-off, talent, and tooling lag the threat.

The source’s broader message is that businesses do not just need more automation. They need controls that can resist machine-speed abuse while still preserving usable access for legitimate customers and staff.

Key questions

Q: How should security teams handle AI-powered bots that target identity and account controls?

A: Security teams should treat AI-powered bots as adaptive identity threats, not just traffic noise. The right response combines stronger account assurance, step-up checks where risk is high, and fraud controls that can change as attacker methods evolve. If the controls only block known scripts, attackers will simply retrain around them.

Q: Why do AI-powered fraud attacks create more pressure on IAM programmes?

A: They compress the time between reconnaissance, credential abuse, and account takeover, which makes static identity controls less effective. IAM programmes have to account for automated login abuse, fake account creation, and recovery-path exploitation as part of the same threat surface.

Q: How do you know if AI-resistant controls are actually working?

A: Look for reduced attacker success, higher retooling effort, and lower abuse conversion rather than only fewer blocked requests. A strong control should make automation expensive, degrade model performance against the challenge, and reduce the number of accounts or sessions attackers can successfully convert.

Q: Who should own response when AI fraud affects customer accounts and identity systems?

A: Ownership should sit across IAM, fraud, and security operations because the issue crosses authentication, account recovery, and abuse monitoring. When those functions operate separately, attackers exploit the handoff gaps. Shared ownership shortens containment and prevents the same pattern from recurring across channels.

Technical breakdown

AI-powered bot attacks and identity abuse

AI-powered bots change fraud economics because they can test accounts, rotate tactics, and scale throughput far faster than manual abuse. In identity terms, that means authentication signals, account recovery paths, and session controls become high-value attack surfaces. The article also distinguishes bots from low-and-slow human fraud farms and from basic scripted automation, which matters because the control strategy differs. Teams that treat all abuse as the same usually overfit to volume and miss adaptation. Practical implication: tune controls for adaptive behaviour, not just request rate.

Practical implication: tune controls for adaptive behaviour, not just request rate.

Why defensive AI adoption lags adversarial AI

The article’s central operational issue is not whether enterprises know AI threats exist. It is whether they can move AI into defense fast enough to matter. Model governance restrictions, approval layers, and AI skill shortages slow deployment, while attackers face none of those constraints. That creates a timing asymmetry: offensive AI can be tested and redeployed continuously, but defensive AI often waits for policy and budget cycles. Practical implication: security programmes should map approval bottlenecks that delay defensive AI use.

Practical implication: security programmes should map approval bottlenecks that delay defensive AI use.

AI-resistant challenges as a fraud control pattern

AI-resistant challenges are designed to force bots into repeated re-evaluation rather than letting them automate through static checks. In practice, this is a moving-target control: the challenge must stay hard enough for machine adversaries while remaining usable for legitimate users. The research suggests that this approach can materially change attacker ROI, which is often the real decision factor for financially motivated fraud. A control that raises the cost of success can be as important as one that blocks every attempt. Practical implication: evaluate controls by how much they erode attacker economics, not only by stop rate.

Practical implication: evaluate controls by how much they erode attacker economics, not only by stop rate.

Threat narrative

Attacker objective: The attacker aims to monetise scale by bypassing account controls, extracting data, and converting automated abuse into direct fraud revenue.

1. Entry occurs through AI-powered bots that automate credential stuffing, fake account creation, and prompt-driven abuse at scale.
2. Escalation happens when attackers use generative AI to adapt payloads, evade static detection, and probe multiple application paths faster than defenders can tune responses.
3. Impact follows when attackers reach account takeover, unauthorised platform replication, data scraping, or fraud losses that hit revenue-driving applications.

Breaches seen in the wild

• DeepSeek breach — DeepSeek breach exposed 1M+ log lines and sensitive secret keys.
• Schneider Electric credentials breach — exposed credentials gave attackers access to Schneider Electric Jira, exfiltrating 40GB.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI-powered fraud is now an identity governance problem, not just a bot problem. Once attackers use AI to scale account abuse, the control plane moves from rate limiting into identity assurance, session trust, and account lifecycle governance. That forces IAM and fraud teams to share responsibility for who or what is actually operating behind each transaction. Practitioners should treat bot resistance as part of identity security architecture, not a separate perimeter layer.

Defensive AI lag is a governance bottleneck, not a technology shortage. The article shows that enterprises are slowed by model governance restrictions, approval layers, and skills gaps while attackers iterate without those constraints. That means the weak point is not awareness of AI risk, but the operational path that delays deploying countermeasures. Practitioners should measure how much time their governance process adds to defensive response.

AI-resistant controls create identity blast-radius pressure on fraud actors. A control that forces repeated machine rework changes attacker economics by increasing cost per attempt and reducing automation yield. That matters because many fraud programmes still optimise for detection after abuse rather than diminishing the scale at which abuse becomes profitable. Practitioners should evaluate whether their controls merely observe AI fraud or actually compress its viable attack surface.

AI enthusiasts are emerging as the most relevant operating model for identity and fraud teams. The article identifies enterprises that use AI across analysis, prediction, automation, monitoring, and response as materially better prepared than peers. That suggests a broader governance shift: the mature programme is not one that uses AI in a single tool, but one that operationalises AI across identity, fraud, and security workflows. Practitioners should benchmark their own operating model against that pattern, not against vendor claims.

From our research:

• 1 in 4 organisations are already investing in dedicated NHI security capabilities, with an additional 60% planning to do so within the next twelve months, according to The State of Non-Human Identity Security.
• Lack of credential rotation is cited as the top cause of NHI-related attacks by 45% of organisations, followed by inadequate monitoring and logging at 37% and over-privileged accounts at 37%, according to the same research.
• That aligns with Top 10 NHI Issues, which helps teams prioritise the controls most likely to reduce machine-identity exposure and abuse.

What this signals

AI-powered fraud is forcing identity programmes to think beyond static authentication and toward adaptive trust decisions. The practical shift is that account security, fraud response, and IAM governance now need to operate as one control system rather than separate workstreams.

Identity blast radius: the useful measure is no longer only whether a control blocks a single attack, but how much abuse it prevents from spreading across accounts, sessions, and recovery flows. That is where the next generation of fraud resilience will be judged.

Enterprises that want to keep pace will need faster internal decision paths for defensive AI, clearer ownership across identity and fraud functions, and controls that can be tuned without creating excessive customer friction.

For practitioners

• Map AI-abuse paths across identity and fraud controls Trace where credential stuffing, fake account creation, prompt abuse, and session hijacking intersect with IAM, customer authentication, and fraud analytics. Prioritise the control points where attackers can reuse the same identity artefacts across multiple abuse patterns.
• Measure defensive AI approval latency Document how long it takes to approve, test, and deploy AI-based detection or response changes. Use that latency as an operational risk metric, because attacker iteration speed is already measured in minutes and hours, not governance cycles.
• Tune controls for attacker economics Assess whether your bot challenges, step-up checks, and account protections increase the cost of automation enough to reduce attacker ROI. Use controlled testing to compare stop rate, user friction, and retooling effort for the attacker.
• Build shared operating ownership between IAM and fraud teams Align identity assurance, account recovery, and fraud response so the same abuse pattern is not handled in disconnected queues. Shared ownership shortens response time when AI-powered attacks move across customer and workforce identity boundaries.

Key takeaways

• AI-powered fraud is already reshaping the identity threat surface, with bots accounting for a large share of enterprise attacks in the survey.
• The main gap is operational readiness: many teams see the threat, but too few can deploy AI-enabled defenses at machine speed.
• Controls that raise attacker cost and compress abuse profitability matter as much as detection, because they change the economics of automation.

Key terms

• AI-powered bot: An AI-powered bot is an automated actor that uses machine-learning or generative techniques to improve speed, variation, and evasiveness during abuse. In fraud and identity contexts, it can mimic user behaviour, adapt to controls, and scale account attacks beyond what static scripts usually achieve.
• AI-resistant control: An AI-resistant control is a defensive mechanism designed to remain effective even when an attacker uses AI to automate or adapt. It does not rely on a single static pattern. Instead, it increases the effort, cost, or inconsistency required for machine-driven abuse to succeed.
• Identity blast radius: Identity blast radius is the amount of damage that can spread when an account, session, or credential is abused. It is a practical measure of how far a compromise can move through customer accounts, recovery paths, transactions, and connected systems before containment stops it.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.

This post draws on content published by Arkose Labs: AI-resistant solutions to defend against AI-powered fraud. Read the original.