AI enthusiasts are reshaping defenses against AI-driven fraud

At a glance

What this is: Arkose Labs' analysis says a small group of enterprises, labelled AI Enthusiasts, are using AI across multiple defensive functions to counter AI-driven fraud and cyberattacks more effectively than peers.

Why it matters: For IAM and security teams, the signal is that AI-driven attacks are pushing detection, response, and governance decisions closer together, especially where fraud controls intersect with model oversight and vendor strategy.

Context

AI-driven fraud is changing the timing and volume of defensive work, which means traditional security operations can no longer rely on slow detection cycles or isolated point controls. In this article, AI is being used as a defensive layer across threat intelligence, monitoring, and response rather than as a narrow automation add-on.

The governance question is not whether enterprises should use AI at all, but how they decide where AI belongs in detection and response, who owns model oversight, and when third-party specialist capabilities make more sense than internal builds. Arkose Labs frames that debate around a cohort of organizations it calls AI Enthusiasts, but the operational issue is broader than the label.

Key questions

Q: How should security teams use AI in fraud and identity defence without losing control?

A: Use AI to improve prioritisation, pattern detection, and response speed, but keep human ownership for high-impact decisions. Security teams should define approval boundaries, logging requirements, and rollback paths before deployment. If the control cannot explain why it acted, or cannot be reviewed after the fact, it is too opaque for identity-sensitive operations.

Q: What happens when AI fraud defence is blocked by internal governance policies?

A: The programme slows down before the threat does. If approval paths, policy constraints, or unclear ownership prevent AI from being used in detection and mitigation, attackers keep the speed advantage and defenders fall back to manual review. That usually means poorer coverage at peak attack volume and weaker response consistency across channels.

Q: What do teams get wrong about building AI fraud controls in-house?

A: They often underestimate the ongoing tuning, monitoring, and governance burden. A working proof of concept is not the same as an operational defence that can withstand adversarial pressure, false positives, and changing fraud methods. If the organisation cannot maintain that pace, the control becomes a maintenance liability rather than a defence.

Q: Should organisations prefer specialist vendors or internal builds for AI-powered fraud defence?

A: Choose the path that can sustain real-time detection, abuse resistance, and governance at attacker speed. In-house builds can make sense when the organisation has strong AI and fraud engineering depth, but specialist vendors are often better when rapid adaptation and continuous tuning are the priority. The decision should be based on operational fit, not control preference alone.

Technical breakdown

How AI changes the fraud detection loop

AI changes fraud defence by shortening the cycle between signal collection, pattern recognition, and response. Instead of waiting for analysts to review logs after an event, models can correlate historical activity with current anomalies and surface likely attack patterns in near real time. That matters most in volumetric fraud and account takeover scenarios, where human review alone cannot keep pace with attack velocity. The technical challenge is not only inference speed, but also model quality, data coverage, and the ability to suppress false positives without creating blind spots.

Practical implication: security teams should treat AI fraud detection as an operational control loop and validate where it accelerates decisions without eroding trust in alerts.

Why model governance becomes part of cyber defence

When AI is used to defend against AI-driven threats, model governance becomes part of the security boundary. That includes approving what data a model can use, defining what actions it can trigger, and limiting where automated decisions can move from recommendation to enforcement. The article points to internal governance policies as a constraint, which is unsurprising because AI systems in security can create their own risk if they are allowed to learn, act, or escalate without clear controls. In practice, governance has to cover data, behaviour, and accountability together.

Practical implication: teams should map defensive AI into governance policy before expanding use beyond simple analysis or triage.

Why specialised vendors still matter in AI-powered fraud defence

Specialised vendors matter because AI-driven fraud defence is a moving target: the attack methods evolve, the tuning burden is constant, and the skill mix needed spans fraud, IAM, and machine learning. Building that capability internally can be viable, but it requires sustained engineering and governance capacity that many enterprises do not have. The article also reflects a broader market reality: defensive AI is increasingly modular, with organisations consuming targeted capabilities rather than building every detection component themselves. That shifts the procurement question from feature parity to control ownership and operational fit.

Practical implication: evaluate whether in-house AI controls can be operated and audited at the same speed as the threat, not just whether they can be built.

Threat narrative

Attacker objective: The attacker objective is to increase fraud success at scale by adapting attack methods faster than defensive review and mitigation can react.

1. Entry begins with AI-assisted fraud campaigns that scale reconnaissance, credential testing, and social engineering faster than manual attack teams can sustain.
2. Escalation occurs when attackers use AI to refine lures, adapt prompts, or iterate on bypass techniques until detection and response windows shrink.
3. Impact follows when account takeovers, MFA compromise, or automated abuse overwhelm traditional response timing and business controls.

Breaches seen in the wild

• Sisense breach 2024: A credential in Sisense's GitLab reportedly opened S3 buckets of customer tokens, passwords and certificates; CISA urged a full reset.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI-driven fraud is forcing security teams to collapse detection and response into a single operating problem. The article shows that the strongest defenders are not treating AI as a separate innovation project but as part of the live fraud control plane. That matters because attack volume, adaptive lures, and real-time abuse all reduce the value of delayed analyst-led workflows. The practitioner conclusion is that fraud defence now has to be measured by reaction speed as much as by coverage.

Model governance is now a security dependency, not just a compliance concern. The article's mention of internal governance policies reflects a wider reality: defensive AI can become constrained before it becomes effective if organisations do not define where automation is allowed to decide, act, and escalate. That is especially true when the same organisation is using AI to counter AI-generated attacks. The practitioner conclusion is that governance must be built into the control design, not added after deployment.

Specialist defensive AI is becoming a procurement pattern because the operating burden is too high for many in-house teams. Arkose Labs' discussion of third-party partnerships points to an industry where speed of adaptation matters more than ownership for its own sake. The issue is not whether an internal team can prototype a model, but whether it can sustain tuning, monitoring, and abuse resistance at attacker pace. The practitioner conclusion is to treat sourcing as part of security architecture, not a separate commercial decision.

Ephemeral attack advantage is the real pressure point in AI-enabled fraud. Adversaries can change tactics quickly, while many defensive programmes still depend on slower review cycles and fixed response playbooks. That imbalance creates a runtime gap between what defenders can observe and what they can stop. The practitioner conclusion is that teams need controls that operate at the same tempo as the attack, not merely controls that look stronger on paper.

AI Enthusiast maturity is a signal that fraud defence is becoming an identity and trust problem as much as a detection problem. Once account takeover, MFA abuse, and adversarial automation sit in the same threat set, the boundaries between IAM, fraud operations, and security analytics blur. That is the direction the market is moving: narrower point solutions will struggle unless they integrate with identity signals, response workflows, and governance. The practitioner conclusion is to design for converged trust enforcement.

What this signals

AI-driven fraud is collapsing the distance between model oversight and security operations. When attackers use AI to scale reconnaissance, lures, and abuse patterns, the defensive programme has to decide faster and with more confidence than traditional review cycles allow. That pushes IAM, fraud operations, and security analytics toward a shared control plane rather than separate queues.

Internal governance can become the limiting factor in defensive AI maturity. If policy design only anticipates model risk in the abstract, teams will find that deployment, escalation authority, and auditability are all constrained at the moment they need speed. The practical shift is to govern defensive AI as an operational control, not a side project.

Specialist sourcing will keep expanding where attack adaptation is faster than internal change management. The market signal is not that enterprises should outsource judgment, but that they will increasingly buy targeted capabilities for detection and mitigation while retaining policy authority in-house. That is a programme design choice, not just a procurement preference.

For practitioners

• Benchmark defensive AI across core fraud functions Assess whether AI is being used for historical analysis, forecasting, automation, 24/7 monitoring, and real-time response, rather than as a single isolated capability.
• Map model governance to response authority Define which AI-driven detections can trigger alerts, containment, or workflow escalation, and which remain advisory until reviewed by a human operator.
• Test AI-resistant controls against account takeover paths Validate challenge, authentication, and mitigation workflows against adversarial automation, reverse-proxy phishing, and other AI-enabled fraud patterns.
• Reassess in-house build decisions for fraud AI Compare the speed of internal model tuning, monitoring, and abuse resistance against specialist vendor options, especially where attacker adaptation is frequent.

Key takeaways

• AI-driven fraud is pushing defenders to use AI as a live control mechanism rather than a narrow analytics layer.
• The operational advantage goes to teams that can combine real-time detection with clear governance over automated response.
• Enterprises should assess whether their AI sourcing model can keep pace with adversarial adaptation and ongoing tuning.

Key terms

• Defensive AI: AI used to help security teams detect, prioritise, or investigate threats more quickly. In practice, it is useful when it reduces analyst time to decision by correlating behaviour across email, identity, and endpoint data, rather than acting as a standalone security control.
• AI-Driven Fraud: AI-driven fraud is deception carried out with artificial intelligence to scale, adapt, or conceal malicious activity. It includes synthetic identities, deepfakes, automated social engineering, and fraud workflows that learn from defenses. In practice, AI increases speed, personalization, and volume while reducing the human effort needed to execute and refine attacks.
• Model Governance: Model governance is the set of controls that decides which foundation models can be used for which agent types and use cases. It links platform choice to security policy, because the model selection influences data exposure, tool behaviour, and the risk profile of the resulting agent.
• Fraud Mitigation: Fraud mitigation is the set of controls used to reduce the likelihood, impact, and spread of fraud. It includes detection, verification, monitoring, escalation, and response measures that work together to stop bad actors while keeping legitimate users and merchants moving through the business.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.