Join our Newsletter — 33% off our NHI Course

Notifications
Clear all

Security data lake layering under SIEM: what it means for SOC teams


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 19785
Topic starter  

TL;DR: Storing every log in a SIEM can force teams into costly, disruptive replacement projects, while Anomali argues that a normalized data lake under the SIEM can move high-volume telemetry out of ingest-priced storage, preserve detection logic, and keep data searchable. The practical shift is architectural: lower SIEM cost without a forklift migration.

NHIMG editorial — based on content published by Anomali: The Security Data Lake Replaces the SIEM's Storage (Without Ripping Out Your SIEM)

By the numbers:

Questions worth separating out

Q: How should security teams reduce SIEM ingestion costs without losing detection value?

A: Teams should move collection, classification, and normalisation upstream so the SIEM receives only the data needed for detection and investigation.

Q: Why do security data lakes help SOC teams more than simply buying a bigger SIEM?

A: Because the core problem is usually storage economics, not detection ambition.

Q: What do teams get wrong when they try to modernize SIEM architecture?

A: They often treat migration as a cutover project instead of a phased routing change.

Practitioner guidance

  • Map telemetry by signal value Classify sources such as DNS, firewall, flow, endpoint, and cloud logs by how often they contribute to detections versus how much ingest they consume.
  • Normalize events to a shared schema Adopt OCSF or an equivalent common model before shifting storage boundaries, so a blocked connection or identity-related event is represented consistently across sources.
  • Migrate one source at a time Use a strangler-fig rollout and validate detection parity before removing each SIEM ingest path.

What's in the full article

Anomali's full article covers the operational detail this post intentionally leaves for the source:

  • Side-by-side storage cost logic for SIEM ingest versus lake retention
  • Stepwise strangler-fig migration sequence for moving logs without a cutover weekend
  • Practical examples of OCSF normalization across firewall, DNS, proxy, and cloud events
  • How to size the business case using ingest reduction, retention savings, and analyst time recovered

👉 Read Anomali's analysis of security data lakes and SIEM storage →

Security data lake layering under SIEM: what it means for SOC teams?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 4 months ago
Posts: 19376
 

Security data lake architecture is becoming a governance decision, not just a tooling decision. Once storage cost starts dictating what telemetry can be retained, detection strategy is constrained by economics rather than risk. That changes how SOC, platform, and identity teams should think about evidence retention, especially where privileged and non-human identity activity must remain searchable across multiple sources. Practitioners should treat data placement as part of control design, not a downstream infrastructure choice.

A question worth separating out:

Q: How do OCSF and a data lake change investigation workflows?

A: They give teams a common event model across sources, which makes cross-platform correlation easier and reduces the need to rewrite detections for each log type. Investigators can query lake data alongside hot SIEM records, so the architecture supports both cost control and retained context.

👉 Read our full editorial: Security data lakes can reduce SIEM storage without cutover risk



   
ReplyQuote
Share: