AI enthusiasts are reshaping defenses against AI-driven fraud

At a glance

What this is: Arkose Labs argues that a minority of enterprises are using AI more broadly across detection and response to counter AI-driven threats and digital fraud.

Why it matters: For IAM, NHI, and autonomous programmes, the article shows how faster attacker automation raises the bar for monitoring, governance, and response across identity controls.

By the numbers:

• AI Enthusiasts are a select group, with just 25% of enterprises fitting this category today.
• Nearly 46% of AI Enthusiasts face hurdles due to internal model governance policies that restrict their use of AI in cybersecurity.
• 50% of AI Enthusiasts acknowledge the shortage of talent with expertise in both cybersecurity and AI.
• 67% prefer partnering with vendors who specialize in AI-powered cybersecurity.

👉 Read Arkose Labs' research on AI, digital fraud, and cyber defences

Context

AI-driven fraud changes the operating tempo for security teams because attackers can scale reconnaissance, impersonation, and abuse faster than manual review cycles can keep up. In practice, that means identity controls have to be evaluated not only for access decisions, but for how quickly they detect and contain abuse once adversarial automation starts probing the environment.

The article is about a subset of enterprises that are using AI defensively across detection, forecasting, automation, and response. The identity governance implication is broader than fraud alone: when attack volume rises, teams need better telemetry, tighter oversight of AI use in security operations, and clearer boundaries between automated defence and human accountability.

Key questions

Q: How should security teams use AI in fraud and identity defence without losing control?

A: Use AI to improve prioritisation, pattern detection, and response speed, but keep human ownership for high-impact decisions. Security teams should define approval boundaries, logging requirements, and rollback paths before deployment. If the control cannot explain why it acted, or cannot be reviewed after the fact, it is too opaque for identity-sensitive operations.

Q: Why do AI-driven attacks change identity governance requirements?

A: Because they compress attacker decision cycles and increase the volume of abuse that identity controls must evaluate. Traditional review cadences assume defenders have time to investigate each event. AI-assisted fraud and account takeover shorten that window, so governance must focus on telemetry quality, containment speed, and accountable automation.

Q: What do security teams get wrong about AI in cyber defence?

A: They often treat AI as a technology purchase rather than a governed operating model. The failure is assuming model adoption automatically improves outcomes. In practice, weak ownership, unclear validation, and poor escalation design can make AI produce faster decisions without making those decisions safer.

Q: Who is accountable when AI-driven defence blocks legitimate users or misses fraud?

A: The organisation remains accountable, not the model. Security, fraud, and identity owners need a shared governance model that defines decision rights, exception handling, and auditability. If an AI system affects access or customer trust, it needs the same accountability discipline as any other identity control.

Technical breakdown

AI-assisted defence against high-volume fraud

Arkose Labs describes AI Enthusiasts as organisations using AI across multiple defensive functions at once, including historical analysis, prediction, automation, continuous monitoring, and real-time response. That matters because AI shifts security from periodic investigation to constant pattern recognition, which is useful when attackers also operate at machine speed. The challenge is not simply using AI, but making sure the model outputs are actionable and bounded by governance, especially where identity abuse and fraud overlap.

Practical implication: teams should map each AI use case to a clear control owner, response path, and audit trail before scaling it in production.

Threat intelligence, detection speed, and response latency

The report links stronger threat intelligence with better detection and faster response. In operational terms, that means the value of AI is often less about perfect prediction and more about compressing the time between signal and containment. For identity teams, that matters because account takeover, MFA compromise, and session abuse are all time-sensitive events. If telemetry is delayed or response orchestration is fragmented, AI tools will not offset the gap.

Practical implication: measure detection-to-containment time for identity abuse scenarios, not just the number of alerts generated.

Model governance and sourcing decisions

A major finding is that nearly half of AI Enthusiasts encounter internal governance constraints, while a majority prefer specialised vendors over in-house builds. That combination shows the real barrier is not enthusiasm for AI, but the operational friction of governing it safely inside security programmes. The underlying issue is accountability: if AI influences fraud defence or identity response, the organisation still needs clear policy, oversight, and validation standards.

Practical implication: align AI sourcing decisions with governance maturity, not with feature claims or team convenience.

Threat narrative

Attacker objective: The attacker seeks to scale fraud, compromise accounts, and exploit identity trust faster than defenders can validate and contain it.

1. Entry begins when adversarial AI or scripted abuse probes identity and fraud controls at scale, looking for weak authentication, weak challenge flows, or compromised accounts.
2. Escalation occurs when the attacker turns a successful login or trust failure into repeated abuse, using automation to test more identities, sessions, or payment flows than manual teams can review.
3. Impact follows when fraud, account takeover, or generative abuse outpaces response, causing loss, operational load, and degraded trust in identity-facing controls.

Breaches seen in the wild

• Cisco DevHub NHI breach — IntelBroker exploited exposed Cisco credentials, API tokens and keys in DevHub.
• DeepSeek breach — DeepSeek breach exposed 1M+ log lines and sensitive secret keys.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI-driven fraud creates a governance problem before it creates a tooling problem. The article shows that enterprises are being pushed to adopt AI because attacker behaviour has become more automated and more adaptive. That changes the control question from whether a security team can detect abuse to whether its governance model can keep pace with AI-assisted attack volume. For practitioners, the real test is whether identity and fraud controls are built for machine-speed escalation.

Internal model governance is now a security constraint, not just a data science concern. Arkose Labs notes that nearly 46% of AI Enthusiasts face hurdles from internal policy restrictions on AI use in cybersecurity. That tells us security teams are not only managing external adversaries, they are also managing their own approval structures. The implication is that AI-enabled defence will stall if model oversight, validation, and accountability are treated as afterthoughts.

Specialised AI security sourcing reflects a market shift toward governed consumption. When 67% of these enterprises prefer specialist vendors, it signals that teams want speed, expertise, and operational assurance rather than bespoke experimentation. The deeper issue is that many organisations lack the combined AI and security talent to safely industrialise these controls. For practitioners, sourcing strategy has become part of the identity and fraud governance design, not a separate procurement exercise.

AI Enthusiasts expose a practical benchmark for the rest of the market. With only 25% of enterprises fitting the category today, the article implies that most organisations are still early in their defensive AI maturity. That creates a useful comparison point for IAM and fraud teams: if response still depends on manual review or disconnected tooling, the programme is already behind the attack curve. Practitioners should treat AI-enabled defence as an operating model decision, not a feature upgrade.

From our research:

• Companies are dedicating an average of 32.4% of their security budgets to secrets management and code security, with US organisations leading at 40.8%, according to The State of Secrets in AppSec.
• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap, according to The State of Secrets in AppSec.
• For a broader identity and secret-risk lens, see NHI Lifecycle Management Guide for the provisioning, rotation, and offboarding controls that reduce exposure over time.

What this signals

Ephemeral AI defence needs lifecycle discipline. As AI tools are pushed deeper into detection and response, the programme has to decide where automation ends and accountable identity governance begins. That includes lifecycle questions such as who can activate, tune, or retire AI-assisted controls, and how those decisions are reviewed in the same way other high-risk identities are governed.

The article also signals that AI security maturity will be measured by operating consistency, not by experimentation volume. Teams that cannot connect model governance, fraud response, and identity telemetry into one control loop will continue to absorb more attack noise than they can safely process. The practical benchmark is whether your current architecture can absorb adversarial load without expanding trust unnecessarily.

For practitioners

• Assess identity response latency Measure how long it takes to detect, triage, and contain account takeover, MFA compromise, and suspicious session activity. Use those numbers to decide where AI can reduce delay and where manual review is still the bottleneck.
• Define governance for defensive AI use Document who approves models, who validates outputs, and who owns the rollback path when an AI control misclassifies user behaviour or blocks legitimate activity. Keep the audit trail tied to identity and fraud outcomes, not just model performance.
• Separate automation from accountability Allow AI to prioritise or enrich alerts, but preserve human decision rights for high-impact actions such as account lockout, challenge escalation, and fraud case closure. That keeps response fast without removing oversight.
• Benchmark against AI-assisted attack volume Review whether your current controls can handle a sustained increase in bot traffic, credential abuse, and replay attempts without degrading user experience. If they cannot, prioritise controls that scale validation rather than adding more manual review.

Key takeaways

• AI-driven fraud forces security teams to treat response speed, not just detection quality, as a core identity control.
• The article’s numbers show that many enterprises are still constrained by governance policy and talent gaps even as attacker automation accelerates.
• Practitioners should design AI defence as a governed operating model with clear ownership, validation, and escalation boundaries.

Key terms

• AI Enthusiast: An AI Enthusiast is an enterprise that uses AI across multiple security functions rather than in a single isolated workflow. In this article, the term signals organisations that rely on AI for analysis, forecasting, automation, monitoring, and response, with governance and talent maturity becoming part of the security outcome.
• Adversarial AI: Adversarial AI refers to AI used by attackers to scale reconnaissance, impersonation, or abuse in ways that overwhelm normal manual review. For defenders, the issue is not just malicious model use, but the way machine-speed behaviour changes the timing, volume, and accuracy demands on identity and fraud controls.
• Identity response latency: Identity response latency is the time between detecting suspicious identity behaviour and containing it. Lower latency matters because account takeover, MFA compromise, and session abuse can compound quickly. When latency is high, even strong detection can fail to prevent fraud, trust erosion, or operational overload.
• Model governance: Model governance is the set of policies, approvals, validation rules, and accountability controls that determine how AI is used in security operations. It matters because an AI tool can only improve defence when its outputs are explainable, reviewable, and aligned to operational decision rights.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.

This post draws on content published by Arkose Labs: The Intersection of AI, Digital Fraud and Cyber Defenses. Read the original.