When the SIEM itself is compromised, the tool you would normally use to investigate is the one place you can no longer trust. An attacker with SIEM access can read credentials that leaked into logs, reuse the connector keys the platform holds into your cloud and SaaS tenants, switch off detections and delete the evidence of everything they did. This playbook covers how the incident shows up, the triage questions to answer first, containment in priority order, a credential rotation plan for every identity the platform holds, rebuilding the platform and its detection content, evidence to keep and notification triggers. It is vendor-neutral and written for SOC leads, detection engineers and incident responders. Prevention is covered separately in Securing the SIEM itself; this page assumes prevention has failed.
Key takeaways
- Investigate from outside the SIEM. Use the platform's own audit log exported elsewhere, cloud provider logs, identity provider sign-in logs and raw archives the SIEM administrators could not alter.
- Every credential the SIEM holds is exposed. Connector keys, ingestion tokens, enrichment accounts, response integration keys and secrets that leaked into indexed logs all need rotation, in a planned order and in one coordinated window.
- Assume detection content was tampered with. Compare deployed rules, suppressions and parsers against the version in your repository before trusting any "no alerts" result.
- Disable response automation early. A compromised SIEM or SOAR that can disable users and isolate hosts can be turned against your administrators.
- Recovery includes a retro-hunt. Once the platform is trusted again, backfill the telemetry gap and hunt across the period when you were blind.
Why a SIEM compromise is different
The UK NCSC introduction to logging warns that an attacker may target the logging service to remove evidence of their actions. A SIEM concentrates that risk: one console controls collection, parsing, detection, retention and deletion for the whole estate, and it usually holds credentials into the systems it watches. In the Microsoft enterprise access model, each plane controls the ones beneath it, and the SIEM touches all of them. Treat a SIEM compromise as a potential compromise of everything it can read from or act on.
The table below is NHI Mgmt Group's view of a common architecture; your platform may name or split these components differently.
| What the attacker can reach | What they can do with it | What it means for the response |
|---|---|---|
| Search console and indexes | Read tokens, passwords and personal data that appear in logs; export in bulk (T1213) | Treat logged secrets as stolen; assess personal data exposure for notification |
| Cloud and SaaS audit log connectors | Reuse API keys or app credentials, or add credentials to the connector app (T1098.001) | Rotate and review in each source tenant, not only in the SIEM |
| Ingestion tokens and collectors | Inject false events, stop forwarding, harvest tokens from collector hosts | Telemetry from the period may be incomplete or forged |
| Detection rules, suppressions and parsers | Disable or narrow detections (T1685), add suppressions for their own accounts | Diff against the repository before relying on alerts |
| Retention and deletion settings | Shorten retention, purge indexes, disable cloud log feeds (T1685.002) | Evidence may exist only in raw archives and source systems |
| SOAR and response integrations | Disable defenders' accounts, isolate hosts, unblock addresses | Pause automation before the attacker uses it |
How the incident shows up
SIEM compromises are often noticed through what stops happening rather than what happens. Signals worth treating as a possible platform compromise include:
- Silence: a sudden drop in events from one or several sources, or a collector that stops sending heartbeats. NCSC advises alerting when log messages stop arriving centrally.
- Content changes outside the pipeline: a high-severity rule disabled or edited, a new suppression on a service account, or a drift alert between the deployed rules and the repository. See the Detection as Code Guide.
- Platform administration you cannot explain: new local users or API keys, role mapping changes, single sign-on settings changed, retention shortened or an index deleted.
- Connector identities acting oddly in source systems: the SIEM's cloud connector signing in from an unexpected network, calling APIs outside its read pattern, or gaining a new secret or certificate (T1078.004, T1098.001).
- Large or unusual exports: bulk searches across restricted indexes, or exports by an account that normally runs a handful of saved searches.
- A vendor notice: for SaaS SIEMs, the provider may tell you its own infrastructure was accessed and ask you to rotate keys. The Sumo Logic breach is an example: customers were asked to rotate API access keys and credentials they had given the platform for other systems, even though no customer data impact was found.
Triage questions
- Which plane is affected? Console user accounts, the platform administration layer, collectors, connectors, or the vendor's own infrastructure. The answer sets the scope of rotation.
- Do we have a trustworthy copy of the platform audit log? If it was exported to separate, immutable storage, start there. If it only lives inside the SIEM, treat it as possibly altered.
- What credentials does the platform hold? Pull the inventory of connector identities, ingestion tokens, enrichment accounts and integration keys. If no inventory exists, build one from the platform's configuration export and from the source systems' app registrations.
- Which secrets were indexed? Run credential pattern searches over the period of exposure, from a trusted copy if possible. Every hit is a secret to rotate at source.
- Can the SIEM act? List every response integration and SOAR playbook that can change identities, endpoints or network controls.
- When did visibility start to degrade? Compare per-source event volumes to their baseline to find the start of the blind window.
- Is this part of a wider intrusion? An attacker rarely targets the SIEM for its own sake. Check the identity provider, privileged accounts and domain controllers through their native logs.
Containment priorities
Contain in this order. Steps 1 to 3 should happen within the first hours; the rest follow once you have a trusted vantage point.
- Set up an independent investigation path. Work from privileged access workstations, query source systems directly (identity provider, cloud provider audit logs, endpoint tools) and move incident communications to an out-of-band channel. See the Out-of-Band Communications for Incident Response Guide.
- Pause response automation. Disable SOAR playbooks and response integrations that can disable accounts, isolate hosts or change firewall rules, and revoke their keys at the target systems. Keep a manual approval path for actions you still need.
- Lock down platform access. Revoke sessions, reset or disable all local accounts except a verified break-glass account, remove unexplained admins and API keys, and restrict console access to a small named response group through single sign-on with phishing-resistant MFA. See the Break-Glass Account Guide.
- Cut the connectors at the source. In each cloud and SaaS tenant, disable or restrict the SIEM's connector identities and review their recent activity there. This stops reuse even if the attacker copied the keys.
- Protect what evidence remains. Freeze retention changes, place holds on indexes where the platform supports it, and copy the platform audit log and configuration export to separate storage.
- Keep visibility running. If you must take the SIEM offline, route critical sources (identity provider, privileged access, endpoint detection) to a temporary store or the native consoles so you are not blind during the response.
Credential rotation plan
Rotate in one planned window rather than piecemeal: the Incomplete rotation breaches guide shows how a credential left behind lets an attacker return. The order below puts identities that can act before those that can only read.
| Order | Credential | Where to rotate | Verify |
|---|---|---|---|
| 1 | Response integration keys (SOAR, endpoint isolation, identity disable) | The target systems that issued them | Old key rejected; new key scoped to defined actions only |
| 2 | Platform admin accounts, API keys and service users | SIEM and identity provider | Admin list matches the approved roster |
| 3 | Cloud and SaaS connector credentials | Each tenant's app registration, IAM role or service account | No extra secrets or certificates on the app; read-only scopes |
| 4 | Enrichment and directory lookup accounts | Directory and asset systems | Interactive logon denied; no privileged group membership |
| 5 | Ingestion tokens and collector keys | SIEM, then each collector host | One token per source; volumes return to baseline |
| 6 | Secrets found in indexed logs | The system that owns each secret | Old value fails; masking rule added so it is not indexed again |
| 7 | Webhook and outbound integration secrets (ticketing, chat) | Destination systems | Test message delivered; old secret rejected |
- Replace, do not just rotate: where a connector used a static key, move it to workload identity federation or a certificate during the rebuild. See the Cross-cloud workload identity federation guide.
- Record each rotation: owner, time, old credential identifier and proof it no longer works. This evidence is what you show a regulator or auditor.
- Use the inventory afterwards: keep the list you built during triage as the platform's NHI inventory. See Credential Rotation at Scale.
Eradication and rebuild
Decide early whether to clean the existing platform or rebuild it. For SaaS SIEMs the vendor owns the infrastructure, so your rebuild is the tenant configuration; for self-hosted platforms you may need to rebuild servers.
| Situation | Clean in place | Rebuild |
|---|---|---|
| Only console user accounts misused, audit log intact | Usually enough, with full rotation and content diff | Not needed |
| Admin layer changed (roles, retention, parsers) | Possible if every change can be identified from a trusted audit log | Preferred if the audit log is incomplete |
| Self-hosted servers or collectors show host-level compromise | Not recommended | Rebuild from known-good images and verified packages |
| Vendor infrastructure compromised | Follow the vendor's advisory and rotate everything you gave them | Consider tenant migration only if the vendor cannot show the cause is fixed |
- Restore detection content from source control, not from the running platform. Redeploy rules, suppressions, parsers and lookups through the pipeline and remove any suppression without an owner and ticket.
- Rebuild roles from the approved design: platform administrator, ingestion, detection, analyst and auditor, with index-level access for sensitive data.
- Reinstall collectors on compromised hosts from verified packages, checking signatures or checksums, and issue new per-source tokens.
- Re-onboard connectors with new identities and the narrowest documented scopes; delete the old app registrations rather than leaving them disabled indefinitely.
- Apply vendor security updates and check whether the entry point was a known vulnerability. If it was, check exposure of other systems running the same product.
Recovery and restoring trust in detection
- Close the telemetry gap: backfill events from source systems and raw archives for the blind window where retention allows. Mark which sources could not be backfilled.
- Retro-hunt: run your high-severity detections and indicator searches over the whole exposure period. NIST SP 800-53 RA-5(8) describes reviewing historic audit logs to see whether a vulnerability was exploited earlier; apply the same idea to the blind window.
- Prove alerts fire: inject test events for critical detections and confirm alerts reach the SOC. Run the SIEM health check before declaring recovery.
- Re-enable automation gradually: bring response playbooks back one at a time with human approval first, and keep a protected list of accounts and hosts automation can never act on.
- Restore from backup carefully: CSF 2.0 RC.RP-03 asks that backup integrity is verified before use. A configuration backup taken after the attacker's changes will reintroduce them.
Evidence to preserve
NIST SP 800-61 Rev. 3 frames incident response through CSF 2.0; outcome RS.AN-07 asks that incident data and metadata are collected with their integrity and provenance preserved. For a SIEM incident, keep:
- Platform audit log: admin actions, role changes, rule and suppression edits, retention and deletion events, API key creation and use.
- Configuration export: roles, users, connectors, rules and parsers as found, before you change them.
- Source system logs for connector identities: sign-ins and API calls by the SIEM's identities in each cloud and SaaS tenant.
- Search and export history: who searched what, and which exports were downloaded.
- Collector hosts: disk and memory images for any self-hosted collector or server showing host-level compromise. See the Evidence Handling and Chain of Custody Guide.
- Vendor communications: advisories, support case records and rotation requests, with timestamps.
Notification triggers
- Personal data: SIEM indexes often hold personal data. If it was accessed, GDPR Article 33 requires notifying the supervisory authority without undue delay and, where feasible, within 72 hours of becoming aware, unless the breach is unlikely to result in a risk to individuals. A processor, including a SIEM provider or MSSP, must tell the controller without undue delay. See the Breach Notification Laws by Country Guide.
- NIS2 entities: for a significant incident, Article 23 of Directive (EU) 2022/2555 sets an early warning within 24 hours of becoming aware, an incident notification within 72 hours and a final report within one month of the notification, as transposed nationally.
- US listed companies: a material cybersecurity incident triggers Form 8-K Item 1.05, generally due four business days after the company determines the incident is material.
- Contracts: if you run the SIEM for customers as an MSSP, check customer notice clauses; their logs and credentials are in scope.
Detection and measurement
Use the incident to put measurements in place that would catch the next one sooner:
- Time from a source going silent to an alert reaching a channel that does not depend on the SIEM.
- Percentage of SIEM credentials (connectors, tokens, integration keys) in the inventory with an owner, scope and rotation date; target 100%.
- Time to rotate every SIEM-held credential in an exercise, measured end to end with verification.
- Number of rule or suppression changes in production without a matching repository commit; target zero.
- Percentage of platform admin actions exported to immutable storage within minutes.
- Credential pattern hits in indexed logs per month and the time to rotate them at source.
Framework mapping
| Playbook step | NIST CSF 2.0 | NIST SP 800-53 Rev. 5 | ATT&CK technique addressed |
|---|---|---|---|
| Triage and scoping | RS.AN-03 | IR-4, IR-5 | T1078 Valid Accounts |
| Containment of platform and connectors | RS.MI-01 | IR-4, AC-2, AC-6 | T1098.001 Additional Cloud Credentials, T1528 Steal Application Access Token |
| Credential rotation and rebuild | RS.MI-02 | IA-5, CM-3 | T1552 Unsecured Credentials |
| Detection content and log integrity | PR.PS-04, DE.CM-01 | AU-9, AU-9(2), AU-9(4), SI-4 | T1685 Disable or Modify Tools, T1685.002 Disable or Modify Cloud Log |
| Evidence handling | RS.AN-07 | AU-11, IR-4 | T1565 Data Manipulation |
| Recovery from backup | RC.RP-01, RC.RP-03 | CP-9, CP-10 | T1565 Data Manipulation |
| Notification and vendor coordination | RS.CO-02, RS.MA-01 | IR-6, SA-9 | T1199 Trusted Relationship |
Common failure patterns
- Investigating inside the compromised console: responders trust "no results" from a platform whose rules and indexes the attacker could edit.
- Rotating in the SIEM but not at the source: a new connector secret is created, but the old one stays valid on the cloud app registration.
- Forgetting logged secrets: tokens in URLs and command lines were readable to the attacker and remain valid after the platform is cleaned.
- Restoring the attacker's configuration: a configuration backup taken after the suppressions were added is restored as "known good".
- Leaving automation live: a SOAR playbook with identity disable rights keeps running while the platform is under attacker control.
- Treating a vendor rotation request as routine: as in the Sumo Logic case, the request is an incident on your side too and needs the same rotation discipline.
Practitioner checklist
- Keep a current inventory of every credential the SIEM holds and every credential it has been given, with where each must be rotated.
- Export the platform audit log and configuration to immutable storage the SIEM administrators cannot alter.
- Document an out-of-band investigation path that does not depend on the SIEM.
- Include a "pause all response automation" step in the SOC runbook, with named owners.
- Rehearse a full SIEM credential rotation once a year and measure the time it takes (our recommendation).
- Keep detection content in source control so it can be redeployed from a trusted copy.
- Keep raw archives of identity and admin logs outside the SIEM so the blind window can be backfilled.
- Pre-agree notification decision makers for personal data in logs, NIS2 and securities disclosure.
Standards and references
- NIST: SP 800-61 Rev. 3 Incident Response Recommendations and Considerations for Cybersecurity Risk Management
- NCSC: Introduction to logging for security purposes
- CISA and ASD's ACSC: Guidance for SIEM and SOAR Implementation
- NCSC: Incident management
- Microsoft Learn: Enterprise access model
- CISA and FBI: Secure by Demand Guide
- EUR-Lex: Directive (EU) 2022/2555 (NIS2)
- GDPR: Article 33 Notification of a personal data breach
Related NHI Mgmt Group resources: Securing the SIEM itself · SIEM integration security guide · SIEM Guide · Incident Response Guide · Service Account Security Guide