Adversarial AI and AI agent access: are your controls keeping up?

TL;DR: Security teams now face two separate adversarial AI problems: attacks against AI systems such as prompt injection, poisoning, and extraction, and AI-enabled attacker tradecraft including phishing, malware adaptation, and LLMjacking, according to Panther. The control gap is no longer model-only; identity hardening, telemetry, and AI-specific detection have become foundational security requirements.

NHIMG editorial — based on content published by Panther: Adversarial AI: Attacks, Risks, and Defenses for Security Teams

By the numbers:

• 28% of breaches are initiated through phishing or social engineering.
• AI-enhanced phishing can boost attack profitability up to 50x.

Questions worth separating out

Q: How should security teams govern AI agents that call internal tools from Kubernetes?

A: They should separate runtime containment from identity governance.

Q: Why do exposed AI secrets create more risk than ordinary cloud credentials?

A: AI secrets often sit between data sources, model access, and downstream business systems, so a single key can expose prompts, outputs, and linked datasets while also enabling access to adjacent services.

Q: What do security teams get wrong about prompt injection defence?

A: They often assume better blocklists will solve the problem, but obfuscation simply changes the shape of the payload.

Practitioner guidance

• Define AI service identities as governed workloads Assign each AI application, agent, or pipeline its own service identity, scope permissions to the minimum model ARNs or API actions required, and remove shared keys wherever possible.
• Centralize AI telemetry with identity context Collect caller identity, model name, token counts, source IP, and response status for every AI request, then route those logs into the SIEM with the same retention and correlation rules used for other privileged workloads.
• Write detections for AI-specific abuse patterns Create rules for anomalous LLM token consumption, prompt injection indicators, credential access on AI service accounts, and suspicious AI agent calls to IAM or metadata services.

What's in the full article

Panther's full blog covers the operational detail this post intentionally leaves for the source:

• Step-by-step control examples for AWS, Azure, and GCP AI services that practitioners can adapt to their own cloud estate.
• Detection logic ideas for prompt injection, anomalous token usage, and AI agent calls into IAM or metadata services.
• Concrete logging fields and schema guidance for building AI telemetry that analysts can query in the SIEM.
• Defensive sequencing for lean teams that need to choose between identity hardening, logging, and model guardrails.

👉 Read Panther's analysis of adversarial AI attacks, risks, and defenses →

Adversarial AI and AI agent access: are your controls keeping up?

Explore further

View Full Forum →  |  NHI Foundation Course →