AI-driven phishing: what changes when agents read email for us?

TL;DR: Generative AI has made phishing more personal, more scalable, and harder to spot by eye, while agentic AI removes the human judgment layer entirely, leaving cryptographic sender verification as the only reliable control when machines process email and act on it, according to Valimail research. Content-based detection still helps, but trust now has to be proven, not inferred.

Editorial analysis by NHI Mgmt Group, based on content published by Valimail: “Generative AI in cybersecurity: The new trust problem”.

Key questions

Q: How should security teams reduce phishing risk when AI makes scam messages more convincing?

A: Teams should stop relying on obvious spelling mistakes and train people to verify the sender, destination, and request through a separate channel.

Q: Why does agentic AI make email authentication more important than content filtering?

A: Because an agent can act on what it reads, while a filter only evaluates what it sees.

Q: What breaks when an AI agent can process email without human review?

A: The human checkpoint breaks first, followed by the assumption that someone will notice a suspicious request before it has an effect.

Practitioner guidance

• Move to DMARC enforcement for all domains Treat p=reject as the baseline for domains that send business mail, because monitoring alone leaves exact-domain spoofing available to attackers.
• Classify autonomous agent actions by approval boundary List which agent actions remain human-approved and which are permitted to execute without review, then remove financial, credential, and data changes from the autonomous set.
• Separate email ingestion from agent execution Ensure that inbound mail can be evaluated by policy before it can trigger routing, payment, or access workflows in an agentic system.

Bottom line: Generative AI has made phishing more convincing and less detectable by eye, so content quality is no longer a reliable trust signal.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →