AI-driven cybercrime is outpacing static email defenses

At a glance

What this is: This is Abnormal AI’s analysis of Anthropic’s report on AI-enabled cybercrime, showing how generative AI helps a lone actor scale extortion, impersonation, and vendor abuse.

Why it matters: It matters because IAM, email security, and fraud controls must now account for AI-assisted deception that can bypass traditional vetting, approval flows, and signature-based detection across human and non-human identity programmes.

👉 Read Abnormal AI’s analysis of AI-enabled cybercrime and vendor fraud

Context

AI-assisted cybercrime is no longer limited by the attacker’s manual skill. The report argues that generative AI can compress reconnaissance, impersonation, and extortion into a much faster operational cycle, which changes the control problem for email, vendor identity, and workforce verification.

For IAM and security teams, the governance gap is not just malicious content. It is the ability of an attacker to appear credible across channels, short-circuit financial approvals, and exploit trust relationships that were designed for human-paced verification and predictable communication patterns.

Key questions

Q: How should security teams handle AI-generated phishing and vendor fraud?

A: Teams should move beyond content-only filtering and evaluate sender behaviour, relationship context, and workflow fit. AI can make messages look polished and credible, but it cannot fully hide when a request is unusual for the account, the recipient, or the business process. The safest control is layered verification for anything that changes money, access, or trust.

Q: Why do AI-assisted attacks bypass traditional vetting so easily?

A: Because vetting often relies on interviews, conversation, and confidence signals that AI can imitate. A low-skill operator can now appear technically competent, persistent, and credible long enough to pass human review. Organisations need separate identity assurance checks, behavioural monitoring, and evidence-based validation for high-trust roles.

Q: What breaks when vendor accounts are used for financial fraud?

A: Approval workflows break when the sender already sits inside a trusted business relationship. A compromised vendor account can make a payment request look routine, which causes employees to bypass normal scepticism. Continuous monitoring of vendor behaviour and independent confirmation of payment changes are the controls that reduce that risk.

Q: How can organisations tell whether behavioural identity monitoring is working?

A: It is working if it flags requests and sessions that are technically valid but inconsistent with normal identity behaviour, device history, or financial workflow patterns. A good programme surfaces anomalies before money moves or access expands, and it produces explainable alerts that analysts can tie back to the actual relationship and activity history.

Technical breakdown

How AI changes the attack surface for email and vendor fraud

Generative AI does not need to create new attack primitives to matter. It can mass-produce believable messages, adapt tone to the recipient, and sustain long conversations that would have been expensive for human attackers to manage manually. That makes phishing, business email compromise, and vendor fraud harder to catch with rules that look for bad spelling, obvious urgency, or known signatures. The real shift is contextual: the message may be synthetic, but the trust relationship it targets is real. Practical implication: move detection away from content alone and toward behavioural signals, relationship context, and workflow deviation.

Practical implication: build detection around behavioural and relationship signals, not just message content.

Why compromised vendor accounts can bypass approval workflows

Vendor accounts sit inside trusted operational paths, so they can inherit credibility that normal email filtering will not question. When an attacker uses that access to request banking changes or urgent payment action, the message can align closely with existing business processes, which makes static approval steps vulnerable to social engineering. The problem is not just the message body. It is the fact that the identity already belongs to a recognised counterparty and may have previously interacted with finance or procurement. Practical implication: treat vendor identity as a monitored access path, not a one-time trust assumption.

Practical implication: monitor vendor behaviour continuously and require independent verification for payment changes.

Behavioural identity monitoring is the control that content filters miss

Behavioural identity monitoring looks for anomalies in login patterns, device posture, location, session timing, and relationship behaviour across accounts. That matters here because AI can imitate competence, but it cannot fully normalise the surrounding behavioural footprint of a legitimate employee or trusted partner. Traditional perimeter tools assume that if credentials are valid, the actor is expected. This report shows why that assumption no longer holds. Practical implication: correlate identity signals with communication and financial workflow context so unusual access and unusual requests are assessed together.

Practical implication: correlate identity telemetry with communication and financial workflow signals before approving high-risk actions.

Threat narrative

Attacker objective: The attacker aims to convert AI-assisted credibility into access, financial leverage, and persistence while avoiding the scrutiny that would normally catch low-skill operators.

1. Entry begins with AI-generated social engineering or impersonation that reaches inboxes, collaboration channels, or hiring workflows and establishes a believable pretext.
2. Escalation occurs when the attacker uses that credibility to harvest credentials, simulate technical competence, or obtain access to vendor and employee accounts.
3. Impact follows when the attacker extorts victims, manipulates financial workflows, or sustains insider-like access long enough to avoid immediate detection.

Breaches seen in the wild

• Cisco DevHub NHI breach — IntelBroker exploited exposed Cisco credentials, API tokens and keys in DevHub.
• ASP.NET machine keys RCE attack — 3,000+ exposed ASP.NET machine keys enabled remote code execution.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI is collapsing the old separation between attacker sophistication and operational reach. The report’s core signal is not that cybercrime has become more automated, but that a single operator can now run a campaign with the tempo and output of a small team. That changes how defenders should think about scale, because the limiting factor is no longer technical expertise alone. Security teams should assume that capability gaps on the attacker side are being filled by models, not people.

Behavioural anomaly detection is now more relevant than content inspection for AI-generated attacks. Static rules work best when threats have predictable markers, but AI-generated lures are built to adapt to tone, timing, and business context. That means the control failure is not merely weak filtering. The deeper problem is that traditional detection assumes messages will look anomalous before they feel normal to the recipient. Practitioners should treat behavioural context as the primary detection layer.

Compromised vendor access creates a short circuit in financial governance. The article shows that trusted third-party identities can move from communication channel to approval path without ever looking obviously malicious. That is a governance problem, not just a phishing problem. Vendor access should be treated as an active identity surface with continuous monitoring, because payment and banking workflows are only as strong as the trust placed in the last sender.

Trust workflows built for humans are failing against AI-assisted impersonation. Traditional vetting assumes that competence, tone, and back-and-forth interaction can be used as proxies for legitimacy. AI breaks that assumption by letting lower-skill operators simulate professionalism at scale. The implication for IAM and fraud teams is that identity assurance must be anchored in observed behaviour and access history, not conversational confidence.

Out-of-context communication is the named concept that security teams should track. When a message, request, or access attempt does not fit the expected relationship, timing, or workflow pattern, the control gap becomes visible. That concept is useful because it bridges email security, vendor management, and identity governance in one analytical frame. Practitioners should map which approvals still rely on contextual trust rather than verified behaviour.

From our research:

• 44% of developers are reported to follow security best practices for secrets management, according to The State of Secrets in AppSec.
• 43% of security professionals are concerned about AI systems learning and reproducing sensitive information patterns from codebases, which reinforces the need for behaviour-aware controls.
• That same research points to the NHI Lifecycle Management Guide as the right next step for teams tightening lifecycle controls around secrets, access, and offboarding.

What this signals

Out-of-context communication: this is the operational signal that matters when AI is used to impersonate employees, vendors, or job candidates. If a request is technically plausible but abnormal for the relationship, the identity programme should treat it as a governance exception rather than a routine transaction.

Enterprise teams should expect more pressure to connect email security, fraud controls, and IAM telemetry into one decision loop. The separate stacks are already too slow for AI-assisted deception, especially when an attacker can move from credibility to action in a single interaction.

The right posture is to make trust conditional on observed behaviour, not on polished language or familiar branding. That means pairing identity monitoring with approval logic and escalation paths that can stop unusual money movement before the workflow completes.

For practitioners

• Shift detection to behavioural context Correlate sender identity, historical relationship, device signals, and workflow timing before trusting high-risk requests. Prioritise alerts where the communication is technically valid but out of context for the relationship or business process.
• Harden vendor payment verification Require independent callbacks or out-of-band confirmation for banking changes, urgent wire requests, and new payment instructions. Do not allow a vendor message alone to satisfy approval criteria, even when the account appears legitimate.
• Review remote hiring and contractor vetting Validate technical competence claims with evidence that is difficult to fake in conversation alone, and tie onboarding decisions to identity checks that are separate from interview channels. Assume AI can amplify presentation skills without proving actual capability.
• Monitor vendor accounts as live identities Continuously watch for unusual login behaviour, device changes, message timing, and approval-path activity on partner accounts. Treat vendor identity as an ongoing governance concern, not a one-time onboarding event.

Key takeaways

• AI is compressing the gap between low-skill actors and high-impact campaigns, which makes static security assumptions obsolete.
• The scale signal is already visible in multi-organisation extortion, fake employment schemes, and compromised vendor abuse.
• Behavioural monitoring, not signature matching, is the control most likely to catch AI-generated deception before it reaches finance or identity workflows.

Key terms

• Behavioral Identity Monitoring: Behavioral identity monitoring is the practice of evaluating logins, devices, timing, and access patterns to determine whether an identity is acting as expected. It is useful when credentials alone are no longer enough to prove legitimacy, especially for vendors, contractors, and remote users.
• Out-of-Context Communication: Out-of-context communication is a message or request that is technically plausible but inconsistent with the normal relationship, timing, or workflow of the sender and recipient. It often signals social engineering, account compromise, or AI-assisted deception before a rule-based filter can catch it.
• Vendor Identity Surface: Vendor identity surface is the collection of accounts, approvals, communication paths, and payment workflows associated with third parties. It matters because a compromised vendor identity can inherit trust across procurement, finance, and operational systems, creating risk that looks legitimate until the final action is already underway.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or governance in your organisation, it is worth exploring.

This post draws on content published by Abnormal AI: AI-driven cybercrime and vendor fraud analysis. Read the original.