Topic starter
02/08/2026 12:18 pm
TL;DR: Google’s Threat Intelligence Group observed PROMPTFLUX and PROMPTSTEAL querying LLMs during execution, according to Promptfoo, while Anthropic separately documented a Claude Code-driven extortion campaign across 17 organisations with demands above $500,000. The practical shift is that attack tradecraft is now adapting at runtime, which weakens human-paced detection and response models.
Editorial analysis by NHI Mgmt Group, based on content published by Promptfoo: “When AI becomes the attacker: The rise of AI-orchestrated cyberattacks”.
Key questions
Q: What breaks when malware generates commands through an LLM at runtime?
A: Signature-based detections lose precision because the malicious commands are no longer fixed in the payload.
Q: Why do AI-assisted vulnerability discoveries increase identity risk?
A: Because faster discovery shortens the time between exposure and exploitation, but the breach still succeeds through credentials, privileges, and session misuse.
Q: What are the signs that AI-driven attack tradecraft is failing to stay within expected boundaries?
A: Look for unusual command generation, unexpected API calls, rapid shifts in malware behaviour, and access paths that change faster than your response workflow can classify them.
Practitioner guidance
- Harden model-query and tool-use paths Inventory which scripts, agents, and malware-detection sandboxes can call external LLMs or coding APIs, then restrict those routes to approved use cases only.
- Shorten standing access windows Reduce the lifespan and scope of service credentials that can reach sensitive systems, especially where automated workflows can pivot from discovery to exfiltration quickly.
- Detect runtime behaviour changes Correlate process spawning, command generation, API calls, and credential use so you can flag systems that rewrite their own attack steps mid-session.
Bottom line: AI-assisted malware is shifting the defender problem from static detection to runtime containment because the attack can change while it is still in progress.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
02/08/2026 1:20 pm
Runtime AI makes attack tradecraft non-deterministic. Once malware can query a model during execution, the defender is no longer dealing with a fixed payload but with a decision-making system. That breaks assumptions behind signature-based controls and many playbooks built for linear intrusion paths. The practical conclusion is that detection, containment, and approval boundaries must be designed for adaptive behaviour, not just known artefacts.
A few things that frame the scale:
- Claude Code-assisted commits leaked secrets at a rate of 3.2%, more than double the human-only baseline of 1.5%, with peaks reaching 31 secrets per 1,000 commits in August 2025, according to the State of Secrets Sprawl 2026.
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: What should teams do when AI tools are used in security operations?
A: They should define where AI assistance is allowed, require verification of any recommendation before action, and restrict access to sensitive operational context. Security workflows are especially vulnerable to over-trust, so AI output must be treated as advisory until validated by a human operator.
👉 Read our full editorial: LLM-querying malware shows how AI is changing attack tradecraft