TL;DR: Storing every log in a SIEM can force teams into costly, disruptive replacement projects, while Anomali argues that a normalized data lake under the SIEM can move high-volume telemetry out of ingest-priced storage, preserve detection logic, and keep data searchable. The practical shift is architectural: lower SIEM cost without a forklift migration.
At a glance
What this is: Anomali says a normalized security data lake under the SIEM can absorb high-volume telemetry, reduce ingest costs, and avoid a disruptive replacement programme.
Why it matters: This matters because SOC and security architecture teams need to balance retention, searchability, and detection fidelity without turning SIEM modernisation into a year-long cutover project.
By the numbers:
- A 100 GB/day Splunk deployment lists around $76,000 a year for the base platform before Enterprise Security is added.
- OCSF joined the Linux Foundation in November 2024.
Context
Security data lake layering is an architectural pattern in which a lower-cost, normalized storage layer sits beneath an existing SIEM. The core governance problem is not whether teams need detection content, but how they prevent ingest-priced storage from becoming the default place every log must live.
For security operations teams, the question is how to preserve correlation, search, and enrichment while reducing SIEM load. This is relevant to SOC, CTI, and security architecture work because the data model, retention policy, and detection workflow all change when hot SIEM data and colder lake data are separated.
The identity angle is indirect but real: the same telemetry architecture that stores firewall, DNS, flow, and endpoint data also shapes how teams investigate account abuse, lateral movement, and machine-generated activity. In practice, the article describes a cyber operations pattern, not an identity programme change.
Key questions
Q: What breaks when a SIEM is forced to store every log source?
A: Cost and operational complexity rise together. Teams end up paying ingest rates for telemetry that contributes little to detection, while parser rewrites, rule migration, and parallel running make replacement slow and risky. The practical failure is treating the SIEM as both the detection engine and the long-term storage layer.
Q: Why does a normalized data lake reduce SIEM risk?
A: It lets teams separate storage economics from detection operations. High-volume telemetry can move into cheaper, searchable storage while the SIEM retains the content that needs fast correlation. That reduces cutover pressure because the SIEM stays in place and the data layer changes underneath it.
Q: What signs show a SOC has outgrown its SIEM storage model?
A: The warning signs are rising ingest spend, repeated pressure to prune logs, long implementation cycles for new sources, and correlation content that is expensive to maintain across multiple telemetry types. If the team hesitates to onboard a source because of cost rather than detection value, the storage model is constraining operations.
Q: How should security teams evaluate security data lakes alongside SIEM investments?
A: Teams should evaluate them by control outcomes, not by storage cost alone. A security data lake is useful when it improves normalization, preserves investigative fidelity, and enables earlier detection logic without locking analytics into one vendor’s schema. If those outcomes are absent, the architecture may only shift costs rather than improve security operations.
Technical breakdown
How a normalized data lake changes SIEM storage economics
A security data lake under the SIEM changes the cost model by moving high-volume, low-signal telemetry out of ingest-priced storage while keeping the SIEM focused on detections and correlations that need fast analyst access. Normalization matters because the lake must preserve field consistency across firewall, DNS, proxy, cloud, and endpoint sources, otherwise the data becomes cheap but unusable. Enrichment at ingest also matters because it keeps the lake from becoming a passive archive. The point is not to replace the SIEM’s detection role but to stop using it as the system of record for everything. Practical implication: route only the telemetry classes that do not need immediate SIEM residency.
Practical implication: route only the telemetry classes that do not need immediate SIEM residency.
Why OCSF makes cross-source detection durable
OCSF gives the data layer a shared schema so one detection logic path can work across different telemetry sources without per-vendor rewrite work. In operational terms, this reduces the chance that a blocked connection, identity event, or endpoint observation is represented inconsistently across tools. That consistency matters because SOC teams lose time when they must reconcile field names, event shapes, and source-specific mappings before they can correlate activity. A durable schema also supports portability, which is useful when teams do not want every future query tied to one SIEM implementation. Practical implication: standardise data representation before expanding the detection surface.
Practical implication: standardise data representation before expanding the detection surface.
How strangler-fig migration avoids a detection cutover gap
The strangler-fig pattern works here because it treats SIEM modernisation as source-by-source substitution rather than a big-bang replacement. Teams keep the existing SIEM running, stand up the new layer beside it, and move one telemetry source and its dependent detections at a time. That avoids the usual failure modes of parser rewrites, manual rule migration, and parallel-run drift. It also creates a cleaner parity test: if DNS detections fire the same way from the lake as they did from the SIEM, the old feed can be retired with less operational risk. Practical implication: migrate by source and prove parity before retiring any ingest path.
Practical implication: migrate by source and prove parity before retiring any ingest path.
NHI Mgmt Group analysis
Security data lake layering is becoming a governance response to SIEM economic sprawl. The article describes a problem many SOCs already feel: storage pressure forces teams to keep paying SIEM rates for telemetry that is rarely used in primary detections. That is not just a tooling issue, it is a control-design issue because the platform economics now shape what gets retained and what gets analysed. Practitioners should treat telemetry placement as a governance decision, not a storage preference.
Normalized telemetry is the real asset, not the lake itself. A lake without schema discipline, enrichment, and deduplication just relocates clutter. The article’s OCSF emphasis shows why shared event models matter: they let teams preserve searchability and correlation across sources instead of fragmenting evidence across vendor-specific formats. For security architecture, the lesson is that data structure determines whether cheaper storage also remains operationally useful.
There is a named control gap here: SIEM-as-storage-system thinking. Many programmes still assume the SIEM must hold everything to remain effective, which turns detection tooling into an expensive archive. That assumption fails when high-volume logs contribute little to alert quality but consume most of the budget. The practical conclusion is to separate detection value from storage value and govern each differently.
OCSF-backed portability creates a better long-term operating model for SOC and CTI teams. When the data model is portable, teams can tune correlation once and reuse it across source classes instead of rebuilding per platform. That reduces dependence on any single ingest pipeline and makes later automation more realistic. For practitioners, the important question is whether their telemetry architecture supports reuse or just re-encodes vendor lock-in.
Security telemetry modernisation is now a prerequisite for agentic SOC design. The article’s final point is the important one for future programmes: an agentic SOC depends on enriched, normalized, queryable data, not on a rigid SIEM ingest boundary. That means teams planning automation should fix the data layer first, then the decision layer. The practitioner conclusion is to design for machine reasoning over normalized evidence, not for alert volume alone.
What this signals
Security telemetry modernisation is increasingly a data architecture problem, not a logging problem. Teams that keep debating SIEM versus replacement often miss the more useful decision point: which sources truly need hot indexing, and which can live in a normalized lake without weakening detection outcomes. The programme implication is to move cost, retention, and query design into the same planning conversation.
Normalized event structures are becoming foundational to machine-assisted operations. If the event model is inconsistent, automation and later agentic workflows inherit the same friction that human analysts already face. Teams should therefore treat schema discipline and enrichment as prerequisites for SOC scaling, not as post-migration clean-up.
For practitioners
- Define telemetry tiering by detection value Classify sources such as DNS, flow, firewall, and endpoint data by how often they drive active detections versus how much they cost to ingest.
- Move one source at a time Use a source-by-source migration sequence so you can validate alert parity before you retire any SIEM ingest path.
- Normalize events to a shared schema Adopt a common event model so fields remain consistent across cloud, network, and endpoint telemetry when data lands in the lake.
- Prove detection parity before cutover Re-run existing correlation logic against lake data and compare outputs before disconnecting the original feed.
Key takeaways
- A security data lake under the SIEM changes the storage and operating model without forcing a disruptive replacement project.
- The main value is not cheap storage by itself, but keeping detections, enrichment, and searchability intact across high-volume telemetry.
- SOC teams should think in source tiers and migration sequences, because parity testing matters more than a single cutover event.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK addresses the attack and risk surface, while NIST CSF 2.0 and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.RM-01 — Risk Management Strategy | The article is fundamentally about balancing security value against SIEM cost and migration risk. |
| PR.PS-01 — Configuration Management | Layering a lake beneath a SIEM changes how telemetry is configured, routed, and retained. | |
| DE.CM-01 — Networks and Network Services Monitored | DNS, flow, firewall, and endpoint telemetry are central inputs to ongoing monitoring. | |
| Recommendation — Align telemetry architecture changes to documented risk appetite and operating priorities. Control data routing and retention settings so storage tiers match detection needs. Preserve monitoring coverage when moving telemetry from SIEM ingest to a data lake. | ||
| CIS Controls v8 | CIS-8 — Audit Log Management | The article focuses on retaining and using logs efficiently across storage tiers. |
| Recommendation — Centralize log collection and retention so audit data stays searchable and cost-controlled. | ||
| MITRE ATT&CK | TA0007;TA0008 — Discovery; Lateral Movement | The telemetry sources discussed are key for investigating discovery and lateral movement behavior. |
| Recommendation — Map layered telemetry to discovery and lateral movement detections across network and endpoint data. | ||
Key terms
- Security Data Lake: A security data lake is a centralised repository for storing large volumes of security telemetry in a queryable form. Unlike a narrow SIEM pipeline, it is designed to keep heterogeneous logs accessible at scale so analysts and automation can correlate identity, endpoint, cloud, network, and application evidence.
- Telemetry Tiering: Telemetry tiering is the practice of assigning different retention or storage levels to events after assessing their investigative value. It helps organisations preserve important evidence in high-fidelity form while reducing cost and noise for routine or low-value data.
- OCSF: The Open Cybersecurity Schema Framework is an open model for describing security events, findings, and objects in a common structure. It lets vendors map their telemetry into a shared language so downstream analytics, investigations, and automation can work across sources more consistently.
- Strangler-fig Migration: Strangler-fig migration is a phased replacement pattern where a new architecture is built beside the old one and workloads move gradually until the legacy path can be retired. For security data platforms, it lowers cutover risk by validating each source and detection before decommissioning the old feed.
Deepen your knowledge
NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It is suitable for practitioners building governance discipline across human and non-human access paths.
Published by the NHIMG editorial team on September 2, 2026.
Updated on October 11, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org