When ransomware reaches the hypervisor, one encryptor run can take down hundreds of virtual machines at once, and the endpoint agents that normally raise the alarm are not running on ESXi. This template gives [Organisation] a hypervisor-specific playbook for VMware ESXi and vCenter encryption incidents. It sits beneath the generic Ransomware Playbook Template, which already covers the ransom decision record, law enforcement reporting and enterprise-wide recovery order; this page adds only what changes when the target is the virtualisation layer. It is written for virtualisation administrators, incident responders, identity teams and backup owners. Replace every [bracketed] placeholder and rehearse it before you need it.
How to use this template
- Run it alongside the generic playbook: declare the incident under the Ransomware Playbook Template, then hand the hypervisor workstream to the Virtualisation Lead using sections 3 to 6 here.
- Print the host inventory: keep an offline copy of every ESXi host, its management IP, vCenter, datastores, root credential vault entry and out-of-band controller (iDRAC, iLO, XCC) address. If vCenter is encrypted you will not be able to look these up.
- Assume the directory is involved: most hypervisor encryption follows domain compromise. Microsoft reported that its incident response engagements involving ESXi more than doubled over three years, with operators abusing Active Directory joined hosts.
- Do not improvise on the datastore: no file is deleted, renamed or "cleaned" on a VMFS or vSAN datastore until forensics has captured it (section 3.4).
- Owner and review: [Head of Infrastructure], reviewed every [12] months and after any vSphere version change.
1. Scope and triggers
1.1 This playbook covers suspected or confirmed encryption, mass power-off or deletion of virtual machines on VMware ESXi hosts, vCenter Server, vSAN and the storage they use, at [Organisation] sites and hosted private clouds. Public cloud workloads use the Cloud Incident Response Guide.
1.2 Triggers (any one opens a SEV1 investigation):
| Trigger | Where you see it | ATT&CK v20 |
|---|---|---|
| Many VMs powered off or killed within minutes, outside a change window | vCenter events, ESXi hostd.log, monitoring | T1529 System Shutdown/Reboot; T1489 Service Stop |
| New file extensions or ransom notes in datastore folders; host client landing page replaced by a note | Datastore browser, users reporting the ESXi web UI | T1486 Data Encrypted for Impact |
| Snapshots deleted, vSphere replication or backup jobs disabled | vCenter tasks, backup console | T1490 Inhibit System Recovery |
| SSH or ESXi Shell enabled on hosts without a change ticket; interactive shell logins | shell.log, auth.log, vCenter alarms | T1021.004 SSH; T1059.004 Unix Shell |
| An AD group named "ESX Admins" created, renamed or given new members | Domain controller security log, identity threat detection | T1078 Valid Accounts (CVE-2024-37085) |
| Syslog forwarding stopped or lockdown mode turned off | SIEM heartbeat, vCenter host configuration | T1685 Disable or Modify Tools |
2. Roles and contacts
2.1 Use the generic roles from the Ransomware Playbook Template. Add the following for this workstream:
| Role | Named owner | Hypervisor responsibilities |
|---|---|---|
| Virtualisation Lead | [Name, deputy] | Host isolation, lockdown, service state, VM inventory, rebuild of hosts and vCenter |
| Storage and Backup Lead | [Name, deputy] | Array and vSAN snapshots, immutable copies, backup proxy and repository isolation |
| Identity Lead | [Name, deputy] | vCenter SSO, AD groups with vSphere roles, service and integration accounts |
| Forensic Lead or retainer | [Firm, contract number] | Host log capture, datastore imaging, encryptor sample handling |
| Hardware vendor and Broadcom support | [Support IDs, phone] | Out-of-band console access, boot media, product guidance |
2.2 Decision rights. The Incident Manager authorises disconnecting the vSphere management network. Powering off hosts or VMs needs the Forensic Lead's agreement unless encryption is still spreading, in which case the Virtualisation Lead may act and record it.
3. Detection and triage
- Confirm scope by host, not by VM: list which hosts and clusters show affected VMs and which datastores they share. Shared datastores mean one compromised host can reach VMs registered elsewhere.
- Check whether vCenter is alive and trustworthy: if vCenter is encrypted or shows unexplained role changes, work host by host through the out-of-band console and the Direct Console User Interface (DCUI).
- Identify the access path: look for SSH sessions, ESXi Shell use, new local accounts, VIB installs, and logins by AD accounts. Check the domain for an "ESX Admins" group: CVE-2024-37085 is in the CISA Known Exploited Vulnerabilities catalogue (added 30 July 2024, known ransomware use). Microsoft found ESXi grants full administrative access to members of that group by name, not by security identifier, and observed Storm-0506, Storm-1175, Octo Tempest and Manatee Tempest abusing it, in some cases before Akira and Black Basta deployment.
- Classify the damage: did the encryptor touch only VM configuration files (.vmx, .vmsd, descriptor .vmdk) or also the large flat or delta disk files? This decides the recovery route in section 6.
3.4 Evidence to preserve before any change:
- Host logs: copy /var/log and /var/run/log (hostd.log, vpxa.log, shell.log, auth.log, vobd.log, syslog.log) and generate a support bundle per host. ESXi keeps logs in memory or on small partitions, so a reboot can lose them.
- vCenter: export events and tasks, the SSO configuration and permission list, and a file-based backup if the appliance is still running.
- Artefacts: ransom notes (keep the replaced web UI index file), a hashed copy of the encryptor binary if found, and a listing of each datastore folder with sizes and timestamps.
- Directory: security events for group creation and membership changes (Windows event IDs 4727, 4728, 4731, 4732, 4737, 4756) covering the ESX Admins group and every group mapped to a vSphere role.
Decision point A: if any trigger in 1.2 is confirmed on two or more hosts, move straight to section 4 for the whole cluster rather than investigating host by host.
4. Containment
- Isolate the management network: block the vmkernel management interfaces and vCenter at the firewall or switch so only a named jump host on an isolated admin VLAN can reach them. Keep VM networks running for unaffected workloads if the Incident Manager agrees.
- Stop remote shell access: stop and disable the SSH (TSM-SSH) and ESXi Shell (TSM) services on every host, from vCenter or the DCUI, and end active shell sessions.
- Enable lockdown mode: set Normal lockdown on hosts that are still managed by a trusted vCenter, so direct host access is refused except for the Exception Users list. Use Strict lockdown, which also disables the DCUI, only where out-of-band console access is confirmed working. Record who is on Exception Users and the DCUI.Access advanced setting.
- Neutralise the AD admin group path: on hosts joined to AD, apply Broadcom KB 369707: set Config.HostAgent.plugins.hostsvc.esxAdminsGroupAutoAdd to false, Config.HostAgent.plugins.vimsvc.authValidateInterval to 90 and Config.HostAgent.plugins.hostsvc.esxAdminsGroup to an empty value, then remove the group's Admin permission and confirm with the host's permission list. No reboot is needed. VMSA-2024-0013 fixes CVE-2024-37085 in ESXi 8.0 Update 3 and states no patch is planned for ESXi 7.0, so the workaround is the only option on 7.0.
- Protect backups now: disconnect backup proxies and repositories that hold vCenter or host credentials, confirm immutability locks on the copies you will restore from, and change backup console credentials from a clean device. See the Backup Identity Isolation Guide.
- Freeze, do not wipe: leave encrypted VMs registered and datastores untouched. Power off a host only if encryption is still running and network isolation has failed, as the CISA #StopRansomware Guide treats powering down as a last resort because it destroys volatile evidence.
Decision point B: if vCenter itself shows attacker activity, treat vCenter SSO as compromised. Disconnect vCenter from identity sources, and manage hosts directly from the jump host until section 5 is complete.
5. Eradication
5.1 Rebuild, do not clean, compromised hosts. Reinstall ESXi from vendor media on each affected host, apply the current patch level, and restore host configuration only from a backup that predates the earliest attacker activity. Confirm no unsigned or unknown VIBs remain on hosts that you keep.
5.2 Rebuild vCenter from a clean appliance and a file-based backup that predates the intrusion, or redeploy and re-add hosts. Review every global and object permission and role before reconnecting identity sources.
5.3 Credential and NHI resets (from a clean admin workstation, in this order):
| Credential | Action | Owner |
|---|---|---|
| ESXi root and local accounts | New unique password per host, stored in the vault; remove unknown local users | Virtualisation Lead |
| vCenter SSO administrator and local SSO users | Reset; review the vsphere.local domain for added users and groups | Identity Lead |
| AD groups mapped to vSphere roles | Remove unknown members; delete or control any ESX Admins group; move to a dedicated, non-default group | Identity Lead |
| Service accounts used by backup, monitoring, orchestration and replication tools against vCenter | Rotate passwords, API tokens and certificates; reduce roles to the minimum privilege | Tool owners |
| Out-of-band controller accounts | Reset; disable shared defaults | Virtualisation Lead |
| Domain Tier 0 | Follow the identity plane steps in the Ransomware Playbook Template | Identity Lead |
5.4 Close the original entry path found in triage, for example VPN without MFA or an exposed management interface, before reconnecting anything. See the Service Account Security Guide for integration accounts.
6. Recovery
Decision point C: choose the recovery route per VM.
| Situation | Route | Caveat |
|---|---|---|
| Only configuration files encrypted, flat disks intact (the ESXiArgs pattern in 2023) | Rebuild VM metadata and re-register the VM. CISA published the ESXiArgs-Recover script, which creates new configuration files from unencrypted virtual disks and keeps the encrypted files in an encrypted_files folder. | CISA states the script comes without warranty and must be reviewed before use. Run it only on copies or after imaging, and scan the guest before reconnecting. |
| Disks encrypted, clean immutable backups exist | Restore to rebuilt hosts on an isolated recovery network | Restore point must predate the earliest attacker activity, not just the encryption |
| Storage array or vSAN snapshots predate the attack and are intact | Revert the volume or clone from the snapshot | Confirm the attacker had no access to the array management interface |
| No clean copy | Rebuild the service from source and data exports; consider the decision record in the generic template | Check public decryptors through law enforcement before any other step |
6.1 Order: isolated admin network and jump hosts; rebuilt hosts with lockdown and syslog forwarding on; vCenter; backup platform with fresh credentials; domain controllers and identity VMs; security tooling; critical services in business impact order.
6.2 Exit check per host: patched, lockdown enabled, SSH and ESXi Shell stopped and set to manual start, syslog arriving in the SIEM, only approved accounts in Exception Users and permissions, and a successful test restore of one VM.
7. Communication and notification
| Audience | When | Owner | Hypervisor-specific content |
|---|---|---|---|
| Service owners of every hosted VM | Within [1] hour | Incident Manager | Which VMs are down, which are untouched, expected restore order |
| Hosting customers or tenants | Per contract, default [24] hours | Communications Lead | Holding statement; no detail of affected hosts |
| Broadcom and hardware vendor support | As needed | Virtualisation Lead | Versions, build numbers, symptoms; never share credentials |
| Law enforcement, national CERT, regulators | Per the generic template's reporting section | Legal | Encryptor hash, ransom note, indicators |
8. Post-incident review
8.1 Within [10] business days, answer: how the attacker reached the management plane; how long SSH or ESXi Shell was open; whether lockdown mode and syslog forwarding were on; which AD groups granted host administration; how long the recovery route decision took; and whether backups held vCenter credentials an attacker could use.
8.2 Track actions to close, then update this playbook and run a tabletop that starts with "vCenter is encrypted". See the Tabletop Exercise Guide.
Implementation guidance
- Prepare the management plane: separate management VLAN, no internet access, vCenter and hosts reachable only from privileged access workstations. Hardening detail is in the VMware ESXi Hardening Guide and the Hypervisor and Virtualisation Security Guide.
- Decide on AD integration: joining hosts to AD simplifies administration but inherits domain compromise. Small estates often manage hosts only through vCenter with lockdown mode on; larger estates that keep AD join should apply KB 369707 settings and use a dedicated, monitored group.
- Forward logs off-host to the SIEM and alert on SSH service start, lockdown changes, mass VM power-off and ESX Admins group events.
- Test: restore a VM from immutable backup to an isolated cluster, rebuild a host from media, and time both. Rollback for containment steps: lockdown and service changes can be reversed from vCenter or the DCUI by an Exception User, so test that path.
Metrics: percentage of hosts in lockdown mode; hosts with SSH running outside change windows; time from first trigger to management network isolation; time to restore a tier 1 VM; percentage of vSphere integration accounts with a named owner and rotation date.
| Framework | References |
|---|---|
| NIST CSF 2.0 | RS.MA-01, RS.AN-03, RS.MI-01, RS.MI-02, RC.RP-01, RC.RP-03, RS.CO-02 |
| NIST SP 800-53 Rev. 5 | IR-4 Incident Handling, IR-8 Incident Response Plan, CP-9 System Backup, CP-10 System Recovery and Reconstitution, CM-7 Least Functionality, AC-2 Account Management, IA-5 Authenticator Management |
| MITRE ATT&CK v20 | T1486, T1490, T1489, T1529, T1078, T1021.004, T1059.004, T1685 |
Standards and references
- Broadcom: VMSA-2024-0013 VMware ESXi and vCenter Server updates
- Broadcom: Knowledge Base article 369707, ESXi Active Directory integration defaults
- CISA: ESXiArgs-Recover
- CISA: #StopRansomware Guide
- CISA: Known Exploited Vulnerabilities Catalog
- Microsoft Threat Intelligence: Ransomware operators exploit ESXi hypervisor vulnerability for mass encryption
- MITRE ATT&CK: T1486 Data Encrypted for Impact
- NIST: SP 800-53 Rev. 5 Security and Privacy Controls
Related NHI Mgmt Group resources: Ransomware Playbook Template · Hypervisor and Virtualisation Security Guide · VMware ESXi Hardening Guide · Backup Security and Immutable Backup Guide · Ransomware Response and Recovery Guide