Notifications
Clear all
Topic starter
27/06/2026 1:14 pm
TL;DR: Cybercriminals are using legitimate AI platforms and dark web models like WormGPT and FraudGPT to generate convincing malicious content at scale, evade detection, and accelerate campaigns, according to Abnormal AI. The trust assumptions behind legacy detection and response need to be re-evaluated before AI-driven abuse becomes the default attack pattern.
NHIMG editorial — here’s why we think this discussion matters
Questions worth separating out
Q: How should security teams respond to AI-generated phishing at scale?
A: Security teams should treat AI-generated phishing as a speed and variation problem, not just a content problem.
Q: Why do malicious GPTs make legacy email controls less effective?
A: Malicious GPTs make legacy controls less effective because they let attackers rewrite the same scam repeatedly until it slips past signature-based filters and human review.
Practitioner guidance
- Map AI-assisted abuse into your identity threat model Identify where malicious content generation can lead into credential theft, token abuse, or privileged action approval.
- Measure detection against novel variants Stop judging controls only by known phishing samples or blocked malicious prompts.
- Tighten identity verification at high-risk interaction points Increase scrutiny on password resets, MFA changes, payment approvals, and privileged requests that often follow convincing lures.
What to expect at the briefing
Abnormal AI's full webinar covers the operational detail this post intentionally leaves for the source:
- How malicious GPTs are used to generate phishing, fraud, and other abuse patterns in practice
- The specific risks associated with WormGPT and FraudGPT as purpose-built cybercrime tools
- AI-driven defense strategies discussed in the session for strengthening detection and response
- The on-demand webinar format and ISC2 CPE eligibility for practitioners who want to follow up
👉 Read Abnormal AI's webinar on malicious GPTs and cybercrime risk →
Malicious GPTs and cybercrime scale: what should teams do now?
Explore further
27/06/2026 2:29 pm
Malicious AI is a governance problem before it is a tooling problem. The article describes how legitimate models and dark web GPTs both reduce the cost of persuasion, which means attackers can industrialise social engineering faster than security teams can review it. That shifts the control burden from message review to identity assurance, campaign correlation, and response speed. Practitioners should treat AI-assisted abuse as an operating model issue, not a content anomaly.
A few things that frame the scale:
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
- Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
A question worth separating out:
Q: How can organisations tell whether their AI threat controls are working?
A: Organisations should measure whether they can detect, classify, and contain novel AI-generated variants before they reach high-risk identity actions. Good performance shows up as faster triage, fewer successful credential-harvest attempts, and tighter coordination between security operations and identity teams.
👉 Read our full editorial: Malicious AI is scaling cybercrime faster than legacy defenses
