SIEM Guide: Use Cases, Tuning and Migration

Most SIEM programmes fail quietly: the platform ingests terabytes a day, the bill grows, analysts drown in alerts, and the one log source that would have shown the intrusion was never onboarded. A security information and event management (SIEM) platform is only as good as the use cases it is built for, the quality of the data it receives and the discipline of the team that writes and tunes its detections. This guide covers use case design, log source onboarding and prioritisation, detection engineering with Sigma, tuning and risk-based alerting, cost and data tiers, and migrating between platforms without losing history. It is vendor-neutral, written for SOC leads, detection engineers and SIEM administrators, and draws on joint guidance from ASD's ACSC, CISA and partner agencies.

Key takeaways

• Start from use cases, not log volume. The joint agencies "discourage logging for the sake of logging" and advise building up data sources gradually; every source should map to a detection or investigation need.
• Not every log belongs in the SIEM. The Priority Logs guidance says the authoring agencies "strongly discourage using a SIEM as the central repository for all logs". Keep detection data hot and route bulk data to cheaper tiers.
• Treat detections as code. Write rules in a portable format such as Sigma, keep them in version control, test them against known-bad and known-good data, and record an owner and expected false positives for each.
• Tune continuously and alert on risk, not single events. Risk-based alerting accumulates low-fidelity signals on a user or host and raises one incident when they cross a threshold.
• Monitor the SIEM itself. ATT&CK T1685 covers interference with "logging pipelines and forwarding mechanisms (e.g., SIEM ingestion)"; a silent source is a detection failure, not a quiet day.

What a SIEM does, and what it does not

A SIEM collects events, parses and normalises them, stores them for search, correlates them against rules and baselines, and hands alerts to analysts through a case workflow. Rule-based event correlation, where a correlation rule joins events from several sources within a time window, is still the core; architectures range from SaaS platforms to self-managed stacks such as ELK (Elasticsearch, Logstash and Kibana). The 2024 joint guidance Best practices for event logging and threat detection, led by ASD's ACSC with CISA, FBI, NSA, NCSC-UK and others, sets four foundations: an "enterprise-approved event logging policy", "centralised event log access and correlation", "secure storage and event log integrity" and a "detection strategy for relevant threats".

The difference between ITDR and SIEM is depth versus breadth: ITDR understands identity systems and can act on them, while the SIEM correlates identity signals with everything else. See the ITDR Guide.

Designing SIEM use cases

A use case is a documented statement of what you want to detect, which data proves it, the detection logic, the expected false positives and the response. Write them from your threat model and recent incidents, then map them to MITRE ATT&CK so coverage gaps are visible.

Each use case should carry a purpose, data requirements with the exact fields, logic, severity, known false positives, a response playbook and an owner. A rule without a playbook produces alerts nobody knows how to close.

Log source onboarding and prioritisation

The 2025 Priority Logs for SIEM Ingestion guidance from ASD's ACSC and partners lists sources in loose priority order: EDR, network devices (firewalls, VPN, proxies, NAC, IDS/IPS), Microsoft domain controllers, Active Directory security logs, Windows endpoints, virtualisation, OT, cloud platforms, containers, databases, MDM, Windows DNS analytic logs, Linux auditing and macOS. It warns that "the volume of firewall or Domain Name System (DNS) logs may overshadow the importance of the information received."

See the Windows Event Log Guide for audit policy settings behind these event IDs.

1. Confirm the use case: name the detections and investigations the source supports before connecting it.
2. Enable the right logging at source: audit policy, cloud diagnostic settings, verbose modes. Collectors cannot forward events that were never generated.
3. Transport securely: the joint guidance recommends "Transport Layer Security (TLS) 1.3"; for syslog, use TLS transport rather than plain UDP 514.
4. Parse and normalise: map fields to one schema (OCSF, the Elastic Common Schema or the platform's own model). CEF and LEEF are common vendor formats; key-value output is easiest to parse, as the guidance notes.
5. Fix time: UTC and "ISO 8601 formatting, with the year listed first", with millisecond granularity where available and synchronised clocks.
6. Validate: generate test events, confirm they arrive parsed within the expected latency, and check that the use case rules fire.
7. Register health baselines: expected daily volume and maximum silence per source, so a drop alerts.

Detection engineering with Sigma

Sigma is an open, YAML-based rule format containing "all the information required to detect odd, bad or malicious behaviour" in logs. A rule declares a logsource (category, product, service), detection selections joined by a condition, a level (informational to critical), a status (experimental, test, stable, deprecated, unsupported), ATT&CK tags and falsepositives. The sigma-cli tool and pySigma backends convert rules into Splunk, Elasticsearch, Microsoft Sentinel and Defender, Grafana Loki and CrowdStrike LogScale queries, which makes Sigma a practical way to keep content portable across a migration.

title: Windows Security audit log cleared
id: 5286b6e2-e075-41e0-8c6f-7b30ecb05fd2
status: test
logsource:
  product: windows
  service: security
detection:
  selection:
    EventID: 1102
  condition: selection
falsepositives:
  - Documented log maintenance
level: high
tags:
  - attack.t1685.005

• Detection as code: rules live in Git, changes go through peer review, and a pipeline converts, tests and deploys them. Rollback is a revert.
• Test both ways: replay attack simulations or recorded malicious events to prove the rule fires, and run it against recent production data to measure false positives before enabling alerts.
• Stage new rules: run them in a monitoring or "test" state that writes results to a separate index or table without paging anyone, then promote once the false positive rate is acceptable.
• Measure coverage: map rules to ATT&CK techniques and to data sources; a technique with a rule but no healthy source is not covered.

Tuning and alert fatigue

Tuning removes noise without removing signal. Every suppression should be specific, documented and reviewed, because broad exclusions are where attackers hide.

• Exclude precisely: suppress a named host plus process plus command line, not a whole subnet or service account. Add expiry dates to exclusions.
• Enrich before alerting: asset criticality, account type (human, service, privileged), and threat intelligence context let rules scale severity instead of firing flat. See the Threat Intelligence Guide.
• Baseline behaviour: the joint guidance notes that "SIEMs can detect anomalous activity by comparing event logs to a baseline of business-as-usual traffic and activity", and that "Behavioural analytics plays a key role" against living-off-the-land techniques.
• Risk-based alerting (RBA): low-fidelity rules write risk events against a user or host instead of alerting; an incident fires when the accumulated score or variety crosses a threshold. Splunk Enterprise Security's default risk incident rules, for example, fire on a risk score of 100 over 24 hours, events from multiple sourcetypes over 3 days, or multiple ATT&CK tactics over 7 days. Other platforms offer equivalent entity scoring.
• Close the loop: analysts tag every closed alert (true positive, benign true positive, false positive); rules with high false positive rates go back to engineering, not into a bigger queue.

Cost, retention and data tiers

Most SIEMs charge by ingested volume, by events per second or by compute. Cost control is a design decision: decide what each source is for, then place it in the cheapest tier that still supports that purpose. The joint guidance recommends "data tiering such as hot and cold storage" and warns that "Default log retention periods are often insufficient", noting that discovery can take "up to 18 months".

• Know what each tier gives up: Microsoft Sentinel's documentation, for example, says its data lake tier offers "Slower queries. Good for auditing. Not optimized for real-time analysis", and moving a table from analytics to the lake means real-time analytics and hunting queries on it stop working.
• Filter and summarise at the edge: drop fields nobody queries, aggregate flow and DNS data, and keep raw copies in cheap storage for forensics.
• Check licensing for security-relevant logs: the Microsoft Storm-0558 Key Breach was first spotted by a US government customer using mailbox access audit events that, at the time, came only with premium logging. In July 2023 Microsoft extended detailed email access logs and more than 30 other log types to Purview Audit (Standard) and raised default retention from 90 to 180 days. Check which security events your own licence tier actually records.
• Protect the store: restrict access, make logs immutable or write-once where possible, and segment the SIEM, which the guidance calls an attractive target.

Choosing a deployment and operating model

• SaaS or self-managed: SaaS removes cluster sizing, upgrades and storage engineering, and suits most organisations. Self-managed platforms (including ELK-based stacks) give control over data residency and cost at scale, but need engineers who can run search clusters as a product.
• Pricing model: ingest-based pricing punishes noisy sources, so tiering and filtering matter most; compute or entity-based pricing punishes heavy searching instead. Model your top ten sources by volume under each option before signing.
• In-house SOC or managed service: a managed detection and response provider can operate the SIEM around the clock, but you still own use cases, log source onboarding and response decisions. Contract for access to raw data, rule logic and exportable detections so you can leave.
• Small organisations: use the SIEM or XDR built into the platforms you already license, onboard identity, EDR, email and cloud control plane logs first, and rely on vendor-maintained content with light tuning.
• Large organisations: a detection engineering team with detection as code, a data pipeline layer that filters and routes before the SIEM, a separate data lake for bulk and historical data, and formal coverage reporting against ATT&CK.

Migrating to a new SIEM

A SIEM migration is a detection programme rebuild under time pressure. The safest pattern is a phased move with a parallel run, history preserved in a searchable form and a defined rollback.

1. Inventory: every source, parser, rule, dashboard, report, SOAR playbook, integration and retention obligation in the old platform, with owners and usage. Retire what nobody uses.
2. Decide on history: keep the old platform read-only for the retention period, export to a data lake or archive with a documented schema, or migrate only the window investigations need. Confirm the legal retention duration before anything is deleted.
3. Translate detection content: rewrite rules as Sigma where possible and convert to the new query language; rebuild parsers to the new schema; record rules that cannot be translated.
4. Dual-feed in waves: forward each source wave (identity and EDR first) to both platforms and compare event counts, parsing and rule results side by side.
5. Parallel run: keep both platforms alerting for an agreed period. Investigate every alert one platform raised and the other missed.
6. Cut over by wave: move analyst workflows, SOAR playbooks and on-call to the new platform once a wave passes its comparison criteria.
7. Rollback: keep forwarding to the old platform until the final wave has run cleanly for a full reporting cycle; switching back is then a routing change, not a rebuild.
8. Decommission: remove collectors, revoke the old platform's service accounts and API keys, and archive its configuration.

Where identity fits

• Identity is the highest-value telemetry: domain controllers, IdP sign-ins, privileged access sessions and cloud role assumptions appear near the top of every priority list. Service principal and managed identity sign-ins are separate log categories in Entra ID and are often not collected. See the Service Account Security Guide.
• The SIEM is full of non-human credentials: collectors, cloud connectors and SOAR integrations hold API keys, service accounts and tokens with read access to every log and sometimes write access to firewalls and directories. Vault them, scope them, rotate them and alert on their use from unexpected hosts. See the Secrets Management Guide.
• Insider and identity context: HR feeds and identity governance data turn raw events into context, such as a leaver's account still signing in. See the Insider Threat and Identity Guide.

SIEM health monitoring and metrics

• Source silence: alert when any source sends nothing for longer than its normal maximum gap, or volume drops sharply against its baseline.
• Parsing failures: track the share of events landing unparsed or with missing key fields.
• Latency: time from event creation to searchability; delayed data breaks correlation windows.
• Tampering: audit policy changes (4719), log clears (1102, T1685.005), cloud logging disabled (T1685.002) and changes to forwarders.

Framework mapping

Common failure patterns

• Collecting everything, detecting little: firewall and DNS volume consumes the licence while identity and cloud control plane logs are missing.
• Nobody notices logging stop: ATT&CK records Salt Typhoon disabling logging and log forwarding to aggregators on network devices, part of how the Salt Typhoon telecom intrusions stayed hidden.
• Retention shorter than dwell time: the joint guidance notes dwell of 70 to 200 days for some malware; 30 days of searchable data cannot scope that incident.
• Vendor rules switched on unchanged: hundreds of default rules fire on normal admin activity, analysts learn to ignore them, and real alerts close as noise.
• Migration by lift and shift: rules copied without testing against the new schema silently match nothing.
• Broad exclusions: a whole service account or admin subnet excluded to stop noise becomes a blind spot.

Practitioner checklist

• Document use cases with data requirements, logic, false positives, playbook and owner.
• Onboard sources by priority: EDR, identity, internet-facing and network edge, cloud control plane.
• Normalise to one schema, use UTC ISO 8601 timestamps and transport logs over TLS.
• Manage detections as code, in Sigma where possible, with tests and peer review.
• Stage new rules silently, then promote; review false positive rates monthly.
• Adopt risk-based alerting for low-fidelity signals.
• Tier storage by purpose and set retention from investigation needs and regulation.
• Alert on silent sources, parsing failures, log clears and audit policy changes.
• Vault, scope and rotate collector and connector credentials.
• Plan migrations with dual feeds, a parallel run, preserved history and a rollback route.

Standards and references

• CISA and ASD's ACSC: Best practices for event logging and threat detection
• ASD's ACSC and partners: Priority Logs for SIEM Ingestion, Practitioner Guidance
• CISA: Guidance for SIEM and SOAR Implementation
• NIST: SP 800-92 Rev. 1 Cybersecurity Log Management Planning Guide (initial public draft)
• MITRE ATT&CK: T1685 Disable or Modify Tools
• SigmaHQ: Sigma Rules
• OCSF: Open Cybersecurity Schema Framework
• Microsoft Learn: Manage data tiers and retention in Microsoft Sentinel

Related NHI Mgmt Group resources: ITDR Guide · Windows Event Log Guide · Incident Response Guide · Threat Intelligence Guide · Digital Forensics (DFIR) Guide