Notifications
Clear all
Topic starter
02/08/2026 12:18 pm
TL;DR: Google's Threat Intelligence Group reported the first malware families observed querying LLMs during execution, while Anthropic separately documented an AI-orchestrated extortion campaign across 17 organisations with demands above $500,000, according to the source article. The practical shift is clear: attackers are now using AI to adapt in real time, which compresses response windows and weakens human-paced detection models.
NHIMG editorial — based on content published by Promptfoo: LLM-querying malware and AI-orchestrated attack tradecraft
By the numbers:
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%).
- When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes.
Questions worth separating out
Q: How should security teams defend against LLM-powered malware that adapts during an attack?
A: Security teams should assume the attack path can change in real time and build controls around behaviour, containment, and identity restriction.
Q: Why do AI tools create NHI governance risk?
A: AI tools create NHI governance risk because they often act with execution authority, data access, and delegated permissions that outlive a single user interaction.
Q: What breaks when malware can adapt its commands in real time?
A: Static signatures, predictable kill-chain assumptions, and scripted response playbooks all become less reliable.
Practitioner guidance
- Log and alert on outbound AI API calls from endpoints and servers Build detections for model endpoints, Hugging Face API traffic, and other LLM service calls originating from scripts, scheduled tasks, and suspicious processes.
- Restrict non-human identities from reaching external model services by default Treat service accounts, workload identities, and automation tokens as high-risk if they can invoke internet-facing AI services.
- Test detection against self-modifying malware scenarios Add purple-team exercises that simulate malware rewriting itself, changing command syntax, or generating one-line execution chains at runtime.
What's in the full article
Promptfoo's full article covers the operational detail this post intentionally leaves for the source:
- The source includes the full breakdown of Google's PROMPTFLUX and PROMPTSTEAL observations, including how each family used LLMs during execution.
- It also describes Anthropic's multi-stage extortion case in more operational detail, including the nine-month attack timeline and the 17-organisation scope.
- The article expands on the difference between AI as operator, builder, and enabler, which is useful if you need to map the threat to your own controls.
- It closes with concrete red-team testing examples for AI systems, which are helpful for teams validating their own guardrails and detection.
👉 Read Promptfoo's analysis of LLM-querying malware and AI-orchestrated attacks →
LLM-querying malware: what it means for defenders and IAM teams?
Explore further
02/08/2026 1:20 pm
LLM-querying malware is the first clear sign that AI is becoming an execution layer for attack operations. This is not a simple case of attackers using chatbots for advice. The model is being called during live compromise to change code, produce commands, and react to the environment. That shifts the threat from static malware to adaptive malware that can alter itself mid-incident. For defenders, the important conclusion is that detection logic built around fixed binaries will increasingly miss behaviour that is generated at runtime.
A few things that frame the scale:
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
- Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation, according to AI Agents: The New Attack Surface report.
A question worth separating out:
Q: What should organisations do first when AI-driven attacks speed up exploitation?
A: Organisations should focus first on identities that already combine privilege, persistence, and secret access. Those are the fastest paths to compromise and the hardest to detect manually. The first 24 to 72 hours should be spent reducing exposure windows, validating revocation, and confirming which agents or service accounts can still reach sensitive systems.
👉 Read our full editorial: LLM-querying malware shows how AI is changing attack tradecraft
