AI cybersecurity threats: what IAM and security teams need to know

TL;DR: AI systems widen the attack surface through prompt injection, jailbreaking, deepfakes, data poisoning, and AI-generated phishing that can bypass conventional controls, according to WitnessAI. The governance problem is not just adversarial content, but the fact that AI systems can be manipulated into taking actions, handling data, or impersonating identities outside intended guardrails.

NHIMG editorial — based on content published by WitnessAI: AI cybersecurity threats, AI agent abuse, and enterprise mitigation strategies

By the numbers:

• When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes and as quickly as 9 minutes in some cases.

Questions worth separating out

Q: How should security teams govern AI systems that can take actions from untrusted input?

A: Security teams should separate untrusted input from action execution, especially where an AI system can call tools, retrieve data, or influence approvals.

Q: Why do deepfakes create a bigger problem than traditional phishing for IAM teams?

A: Deepfakes weaken the signals people use to approve access changes, reset credentials, or authorise transactions.

Q: What do organisations get wrong about AI-generated phishing and impersonation?

A: They often treat it as a messaging problem instead of an identity problem.

Practitioner guidance

• Classify AI-exposed workflows by trust boundary Map where AI systems consume untrusted input, make decisions, or trigger downstream actions.
• Restrict model-to-tool permissions tightly Limit which tools, datasets, and actions an AI system can reach, and require separate approval for high-risk operations.
• Harden human verification for approval paths Move beyond voice or message realism when approving resets, payments, or access changes.

What's in the full article

WitnessAI's full article covers the operational detail this post intentionally leaves for the source:

• Specific examples of prompt injection and jailbreak patterns that defenders can test against their own AI stack
• Detailed countermeasures for AI-specific phishing, deepfakes, and model abuse across enterprise workflows
• Guidance on how to update incident response playbooks for AI-enabled attack paths
• A vendor view of runtime security and observability for models, applications, and agents

👉 Read WitnessAI's analysis of AI cybersecurity threats and AI agent abuse →

AI cybersecurity threats: what IAM and security teams need to know?

Explore further

View Full Forum →  |  NHI Foundation Course →