By NHI Mgmt Group Editorial TeamDomain: Breaches & IncidentsSource: SwarmneticsPublished November 11, 2025

TL;DR: Google Threat Intelligence Group says it has documented the first known uses of Gemini in live malware attacks, alongside multiple AI-enabled malware families that can generate scripts, alter obfuscation, and support phishing, reconnaissance, and ransomware workflows during an attack, according to Swarmnetics. Static, rules-based defence is now competing with adversarial tooling that can adapt mid-execution.


At a glance

What this is: This analysis says AI-assisted malware is moving from support functions into live attack workflows, with early examples showing on-the-fly script generation and obfuscation changes.

Why it matters: That shift matters because IAM, PAM, and NHI teams may face faster credential abuse, more adaptive phishing, and less predictable attack paths that outpace static controls.

👉 Read Swarmnetics's analysis of LLM-powered malware and AI-enabled attack adaptation


Context

LLM-powered malware changes the security problem from detecting known malicious strings to dealing with software that can rewrite parts of its own attack path while it runs. In practice, that weakens the value of static signatures and forces defenders to rely more heavily on behavioural controls, identity restrictions, and runtime telemetry. The primary keyword here is LLM-powered malware, but the governance question is broader: what happens when offensive tooling can adapt faster than your control review cycle?

The article also has a genuine identity angle because adaptive malware commonly targets credentials, tokens, and phishing entry points before expanding into privileged systems. That creates overlap between human identity, NHI secret exposure, and access governance, especially where attackers can use stolen access to call tools, enumerate resources, or pivot into cloud environments. For practitioners, this is not an abstract AI issue, it is an access-control and containment problem that touches IAM, PAM, and NHI oversight.


Key questions

Q: How should security teams defend against LLM-powered malware that adapts during an attack?

A: Security teams should assume the attack path can change in real time and build controls around behaviour, containment, and identity restriction. Static signatures still matter, but they are no longer enough on their own. Prioritise runtime monitoring, tight privilege boundaries, and fast isolation of suspicious sessions or tooling.

Q: Why do exposed credentials matter more when attackers use AI-assisted malware?

A: Exposed credentials give adaptive malware a foothold it can use to generate scripts, explore systems, and change tactics faster than manual attackers. Once inside, the model-assisted workflow can amplify every weak access decision, especially where service accounts or tokens have broad scope. That makes identity hygiene a force multiplier for defence.

Q: What do security teams get wrong about AI-driven ransomware?

A: They often focus on whether the malware is novel instead of whether the operator behaviour is familiar. AI changes the economics of attack creation, but the same identity, access, and workflow patterns still matter. If you only look for known hashes or static indicators, you miss the abuse path that makes the malware effective.

Q: Who is accountable when AI tools are abused to support malware operations?

A: Accountability sits across AI governance, security operations, and identity ownership. Teams that approve models, expose them to users, or connect them to tools need documented controls for abuse detection, access restriction, and incident response. Where AI assistants are integrated into workflows, governance must cover both the model and the permissions around it.


Technical breakdown

How LLM-powered malware changes the attack loop

Traditional malware tends to follow a prebuilt sequence, which defenders can model with signatures, detections, and known indicators. LLM-powered malware introduces a different pattern: the attacker can ask the model to generate code, adjust obfuscation, rewrite phishing text, or vary execution details while the attack is in progress. That makes the malware less deterministic and reduces the usefulness of fixed rules that assume the payload will stay constant. In early-stage implementations, the model still depends on attacker prompts and tool access, but the important change is runtime adaptability rather than fully autonomous behaviour.

Practical implication: expand detections beyond static hashes and signatures to behavioural telemetry that can spot adaptive tool use.

Why guardrails and jailbreak resistance matter to security teams

The report describes threat actors succeeding by reframing malicious requests as benign tasks such as capture the flag exercises or academic work. That shows a familiar AI security failure mode: policy enforcement at the model boundary is only as strong as the prompt filtering, abuse detection, and safety controls around it. If a model can be induced to assist in malware creation or obfuscation, then the risk is not simply misuse by a user, but model-mediated acceleration of attack development. For defenders, that turns model guardrails into a frontline security dependency, not a product feature detail.

Practical implication: treat model safety controls as part of the control stack and test them against realistic abuse prompts.

Why identity controls still sit at the centre of AI-enabled intrusion

Even when the malware is AI-assisted, most real attacks still need an entry point, credentials, or a trusted session to reach useful systems. That is where IAM and NHI governance remain decisive. Compromised accounts, exposed API keys, and over-permissioned service identities create the access path that AI tooling can then exploit more efficiently. In that sense, the new capability does not remove identity risk, it amplifies the consequences of weak identity hygiene by making reconnaissance, lateral movement, and payload variation faster and cheaper for the attacker.

Practical implication: reduce standing access and tighten NHI credential scope before adaptive tooling can exploit it.


NHI Mgmt Group analysis

Adaptive malware creates a control problem, not just a detection problem. The central change is that offensive tooling can now modify its own behaviour during execution, which weakens any defence strategy that depends on a stable attack pattern. That shifts emphasis toward runtime policy, behavioural detection, and identity containment rather than static malware signatures alone. Practitioners should assume attackers will probe controls in real time and should design for interruption, not certainty.

Identity exposure remains the most practical accelerator for AI-enabled attacks. LLM tooling can generate code and alter obfuscation, but it still needs access to make those actions matter. Exposed credentials, service accounts with excessive privilege, and weak session boundaries become higher-value because they allow adaptive tooling to move faster once it lands. That makes IAM and PAM the control plane that determines whether AI-assisted intrusion stays local or becomes enterprise-wide.

Static security operations are now mismatched to adaptive attack tooling. The article’s core signal is not that AI has replaced malware, but that malware is becoming more iterative and context-aware. That erodes the assumption that yesterday’s detections will describe today’s attack path. Security teams should treat this as a prompt to prioritise behavioural baselines, containment controls, and faster identity revocation when anomalies appear.

AI guardrails are now part of enterprise cyber resilience. If threat actors can coerce general-purpose models into helping with malicious tasks, the safety boundary around AI systems becomes a security dependency. That matters beyond the model layer because AI abuse can accelerate the very attack phases that identity and infrastructure controls are meant to stop. Practitioners should align model governance with access governance, because neither layer is sufficient on its own.

From our research:

  • 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
  • 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
  • For a related control view: Review OWASP Agentic AI Top 10 for the abuse paths that tend to emerge when AI systems are allowed to call tools.

What this signals

Adaptive attack tooling raises the value of identity containment. When malware can rewrite its own behaviour, the best response is to make every credential and session as small and short-lived as possible. That does not eliminate AI-enabled threats, but it reduces how far a single compromise can move before controls terminate it. For teams building against this risk, the practical goal is to make adaptation expensive for the attacker.

AI governance now overlaps with access governance in a way most programmes have not fully operationalised. If an assistant can be coerced into helping with malicious tasks, then model access, tool access, and identity access have become one control surface. Linking model policy to identity enforcement is where this post moves from theory into programme design, and the relevant policy baselines are already being discussed in the NIST AI 600-1 Generative AI Profile and the OWASP Agentic AI Top 10.

LLM-mediated abuse is likely to expose the same weak spots repeatedly. The organisations most at risk are those with broad service account scope, poor token governance, and limited visibility into what AI tools can reach. The operational signal to watch is not just malicious content, but unauthorised action chains that begin with legitimate access and end in unexpected execution.


For practitioners

  • Tighten runtime controls around AI-accessible tooling Block or constrain outbound model calls, command execution, and script generation paths where malware could reach them, then monitor for unusual prompt-to-action chains in production environments.
  • Reduce the value of stolen identity material Prioritise least privilege, short-lived access, and aggressive revocation for service accounts, API keys, and session tokens so an AI-assisted attacker has less room to pivot after initial compromise.
  • Shift detections toward behaviour, not signatures Tune SOC use cases for rapid obfuscation changes, unusual code generation, and abnormal repository or environment enumeration rather than relying on fixed malware indicators.
  • Test AI safety controls against abuse prompts Validate whether your approved models and hosted assistants can be induced to support malicious workflows using framed requests, and record how quickly those requests are blocked or surfaced.

Key takeaways

  • LLM-powered malware changes the attack tempo by making malicious behaviour adaptive during execution, which reduces the value of static detections.
  • Identity exposure remains the enabling condition, because stolen credentials and over-privileged accounts give AI-assisted attackers the access needed to turn adaptation into impact.
  • Defence now needs runtime containment, model abuse testing, and tighter privilege boundaries so that AI-enabled intrusions fail before they can spread.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

OWASP Agentic AI Top 10 and MITRE ATLAS address the attack and risk surface, while NIST AI RMF, NIST CSF 2.0 and NIST SP 800-53 Rev 5 set the governance and control requirements practitioners need to meet.

FrameworkControl / ReferenceRelevance
OWASP Agentic AI Top 10OWASP Top 10 for Agentic ApplicationsThe article centres on AI abuse, tool misuse, and adaptive agent-like attack behaviour.
NIST AI RMFMANAGEAdaptive AI attack use requires ongoing risk treatment and operational controls.
MITRE ATLASTA0002 , Execution; TA0006 , Credential Access; TA0008 , Lateral MovementThe attack pattern spans execution, credential abuse, and movement across environments.
NIST CSF 2.0PR.AC-4Least-privilege access is central when AI tooling can accelerate abuse of valid identities.
NIST SP 800-53 Rev 5SI-4Monitoring for anomalous behaviour is essential when payloads can change during execution.

Use ATLAS to map AI-enabled intrusion behaviour to execution, credential access, and lateral movement.


Key terms

  • LLM-powered malware: Malware that uses a large language model during an attack to generate code, rewrite text, alter obfuscation, or adapt tactics. The model does not replace the attacker, but it can increase speed, variation, and resilience while the intrusion is underway.
  • Adaptive attack tooling: Attack tooling that changes its behaviour in response to defensive measures or environment conditions. In AI-assisted campaigns, this often means generating new scripts, adjusting payloads, or varying execution patterns so static detections become less reliable.
  • Model guardrails: Safety and abuse controls designed to prevent a model from helping with harmful requests. In practice, guardrails include prompt filtering, policy enforcement, tool restrictions, and monitoring for misuse patterns that indicate the model is being steered into malicious support.
  • Identity Containment: The practice of revoking or constraining an identity’s ability to act after compromise is suspected. It goes beyond isolating the device and includes session termination, token revocation, privilege reduction, and validation of what the identity can still reach.

What's in the full analysis

Swarmnetics's full analysis covers the operational detail this post intentionally leaves for the source:

  • How Google Threat Intelligence Group characterises the first known Gemini-involved malware activity and the specific limitations observed in early deployments.
  • The report's breakdown of AI-enabled malware families, including where they support phishing, reconnaissance, obfuscation, and ransomware workflows.
  • The examples of jailbreak and abuse prompting that showed how threat actors got model assistance past safety boundaries.
  • The vendor's discussion of how defenders can adapt monitoring and response to the emerging AI attack cycle.

👉 Swarmnetics's full post covers the attack examples, model abuse patterns, and defender implications in more detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps security practitioners translate identity controls into practical programme decisions across access, rotation, and oversight.
NHIMG Editorial Note
Published by the NHIMG editorial team on July 14, 2026.
NHI Mgmt Group — the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org