AI-driven social engineering is outpacing fraud controls in finance

At a glance

What this is: Alloy argues that generative AI is accelerating phishing, vishing, smishing, and deepfake-enabled scams, while exposing the limits of isolated identity checks and static fraud rules.

Why it matters: This matters because fraud teams and IAM-adjacent controls now have to evaluate behaviour, trust signals, and step-up decisions together rather than treat authentication as proof of intent.

By the numbers:

• 85% of Americans worry that advances in AI technology are making scams harder to detect.
• 27% of Americans have experienced an AI-generated scam personally, or know someone who has.
• Per month, Americans received an average of 63 spam texts each.
• Approximately one-fourth of Gen Z and Millennial consumers have lost $5,000 or more to fraud.

Context

AI has lowered the cost of social engineering and raised the quality of deception at the same time. When callers, texts, and videos can be spoofed convincingly, the security problem is no longer only message filtering, it is trust validation under urgency, especially in financial services where a legitimate customer can still be manipulated into a harmful action.

For IAM, fraud, and identity-verification teams, the important governance question is how to combine identity evidence, device signals, and behavioural context before authorising a transaction or account change. The article’s core theme is that authentication alone does not prove intent, and that gap is exactly where scam losses now grow.

Key questions

Q: What breaks when a real customer is manipulated by an AI-enabled scam?

A: The failure is assuming that authenticated access equals legitimate intent. In AI-enabled social engineering, the customer may pass login or step-up checks while still being pressured into a harmful transfer or account change. Controls fail when they do not combine identity, device, and behavioural context before approval.

Q: Why do AI-generated scams still succeed even when users know about phishing?

A: Because awareness alone does not stop pressure, familiarity, and speed from shaping decisions. AI makes it cheaper to copy a trusted voice, face, or message, while urgent framing shortens the time available to check. The result is not better persuasion in the abstract, but faster and more convincing manipulation of normal business behaviour.

Q: What are the signs that scam detection is too dependent on static rules?

A: Common signs include missed deepfake lures, repeated false negatives on unusual transfers, and alerts that only fire after account data has already changed. If analysts only see fragments, the system is likely relying on isolated signals instead of a joined behavioural view.

Q: Should financial institutions use step-up verification or behavioural scoring first?

A: They should use both, but for different purposes. Behavioural scoring identifies when a request looks inconsistent with prior activity, while step-up verification interrupts the transaction and forces the customer to re-confirm the action. The best programmes combine early risk detection with friction at the point of execution.

Technical breakdown

Why AI makes social engineering harder to filter

Generative AI reduces the effort needed to create convincing phishing, vishing, smishing, and deepfake lures. Large language models let attackers vary tone, imitate known relationships, and tailor messages to public details pulled from social media or breached data. That changes the economics of fraud: one operator can run many believable interactions at once, and the signal quality improves enough to beat simple keyword filters or static playbooks. In practice, the attacker does not need perfect impersonation. They only need enough realism to trigger urgency, trust, or fear before the target pauses to verify the request.

Practical implication: Treat AI-assisted impersonation as a scale problem, not just a content problem, and move controls closer to the action point.

Why authenticated customers can still be fraud risks

The article’s most important control insight is that a real, authenticated customer can still be acting under social engineering pressure. That means identity proofing, login success, and even step-up completion do not by themselves establish that the transaction is legitimate. The fraud decision must consider whether the current request fits prior behaviour, device posture, contact-data changes, and recent account activity. This is where lifecycle signals matter. A sudden phone number change, unusual login time, or modified security setting can indicate coercion or external manipulation even when the account holder passes authentication.

Practical implication: Correlate identity and behaviour signals before approving high-risk transactions or profile changes.

How orchestration and scoring change fraud operations

Alloy’s discussion points to a broader pattern in fraud operations: point solutions see fragments, while scam detection needs a composite view. Orchestration layers assemble identity, device, and behavioural data into one decision path, and scoring models turn those inputs into a ranked risk view that can trigger review or friction. The technical value is not the score itself, but the ability to compare current behaviour against expected patterns in near real time. For investigators, agentic assistance can summarise risk drivers and likely next steps, reducing the delay between anomaly detection and containment.

Practical implication: Design fraud workflows so the first useful decision is automated, and the final decision is explainable to an analyst.

Threat narrative

Attacker objective: The attacker’s objective is to manipulate a legitimate customer into authorising a transfer or divulging sensitive account information that can be monetised immediately.

1. Entry begins with spoofed caller ID, phishing, smishing, or a deepfake voice that creates a believable pretext for contact.
2. Credential or trust signals are then abused when the victim is pressured to disclose codes, confirm account details, or authorize a transfer under manufactured urgency.
3. Impact occurs when money moves or account settings change before the victim can verify the request, creating direct financial loss and downstream recovery work.

Breaches seen in the wild

• Zacks breach claim 2025: A hacker leaked 12 million Zacks accounts in 2025, claiming domain admin access in 2024; HIBP verified the data, Zacks has not confirmed.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI has turned social engineering into a workflow problem, not just a messaging problem. The article shows that attackers now use automation, deepfakes, and generative text to scale persuasion across channels. That means the defensive unit of analysis is no longer the email or call, but the full trust journey from first contact to authorised action. Practitioners should govern scam risk as an identity and decisioning problem, not as a pure communications-filtering issue.

Authentication success is no longer a sufficient fraud signal. A legitimate customer can complete login, MFA, or verification while still being under active manipulation. That creates a governance gap between identity proof and behavioural intent, which is where fraud now concentrates. Financial institutions need controls that measure whether the request fits the customer’s normal pattern, because validated identity does not guarantee valid intent.

Behavioural context is becoming the new control surface for financial trust. The article’s strongest signal is that isolated checks miss the interaction between device, account history, and transaction timing. Behavioural trust gap: the failure mode where a real customer is treated as low risk even as surrounding signals point to coercion or impersonation. Teams should use this concept to redesign decisioning, review, and customer-friction policies around context, not just credentials.

Agentic AI is starting to reshape both sides of fraud operations. Attackers use it to generate scripts, scale outreach, and refine impersonation. Defenders use it to condense risk signals and accelerate review. The governance issue is not whether AI is helpful, but whether human oversight still has enough context to intervene before money leaves the account. Practitioners should treat AI as an operational amplifier that must be bounded by explicit decision thresholds and escalation rules.

Fraud prevention now sits at the intersection of IAM, PAM-adjacent step-up controls, and financial decisioning. When a customer change or transfer is suspicious, the right response is not only to verify identity but to interrupt the action path with context-aware friction. That is a governance pattern identity teams should recognise, because the same logic increasingly applies to high-risk account recovery and delegated access flows. Practitioners should align fraud controls with identity governance rather than leaving them as separate silos.

From our research library:

• U.S. fraud losses are projected to reach $40 billion by 2027.

What this signals

Behavioural trust gap: financial fraud teams now need controls that distinguish a verified customer from a manipulated customer, because the attack often succeeds after authentication rather than before it.

Identity and fraud programmes should converge around decisioning, not remain separate queues. When social engineering is AI-assisted, the most useful control is the one that can explain why a request looks abnormal before funds move.

For practitioners

• Implement behaviour-aware transaction gating Block or step up high-risk transfers and profile changes when device, location, contact data, or login timing diverge from the customer’s normal pattern.
• Correlate identity and behavioural signals Join onboarding, login, and transaction telemetry so a legitimate authentication event can be weighed against contextual fraud indicators before approval.
• Add friction at the point of urgency Use step-up verification, callback checks, or out-of-band confirmation when requests involve unusual payment instructions or account recovery.
• Tune analyst workflows for AI-assisted scams Prioritise cases where deepfake indicators, sudden contact-data changes, or atypical access patterns indicate external manipulation rather than customer intent.
• Reduce reliance on isolated identity checks Rework decision rules so passing authentication does not automatically clear a transfer, a new beneficiary, or a security-setting change.

Key takeaways

• AI has made scam content cheaper to produce and harder to distinguish from legitimate communication, which shifts fraud defence toward context and timing.
• The strongest signal in the article is that authenticated customers can still be manipulated into authorised loss, so identity success is no longer enough on its own.
• Financial institutions need joined identity, device, and behavioural controls, plus step-up friction, to interrupt scams before the transaction completes.

Key terms

• Social Engineering: Social engineering is the use of deception, urgency, and authority to persuade a person to reveal information or take a risky action. It targets human decision-making rather than software defects, and often turns legitimate identity workflows into the attack path.
• Deepfake-based impersonation: A fraud technique that uses synthetic audio, video, or both to make an attacker appear to be a trusted person during a live interaction. The tactic exploits human trust in familiar cues and often aims to trigger urgent actions such as payments, resets, or access changes before verification is challenged.
• Behavioural Risk Signal: A behavioural risk signal is an observable action or pattern that suggests increased likelihood of unsafe security behaviour. Examples include repeated simulation failures, risky handling of data, or ignoring recommended actions. Used well, these signals inform targeted guidance rather than broad, generic awareness campaigns.
• Step-Up Verification: Step-up verification is a stronger identity check applied when risk increases, such as during password reset, device change, or privileged access request. It uses higher-assurance signals than a static question, such as device possession, authenticated context, or approved administrative review.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It gives identity and security practitioners a common operating model for controlling high-risk access across the full lifecycle.