Start by deciding which telemetry must support immediate alerting and which must support long-term analysis. Route high-value events to the SIEM, keep raw and enriched data in the lake, and preserve identity context early so investigations do not depend on reconstruction after the fact.
Why This Matters for Security Teams
A hybrid SIEM and data lake design is about more than storage efficiency. It determines whether the security team can detect threats quickly, preserve evidence for later analysis, and retain enough context to understand who or what acted. The SIEM is best used for time-sensitive correlation and alerting, while the lake supports retention, enrichment, and retrospective hunting. That split only works if logging scope, data quality, and identity context are defined up front.
Security teams often get this wrong by treating the lake as a cheaper archive for whatever the SIEM cannot handle. That approach creates blind spots because critical events arrive without normalized fields, asset metadata, or identity attribution. A better pattern is to align telemetry classes to use cases, not tool boundaries, and to anchor control decisions in sources such as NIST SP 800-53 Rev 5 Security and Privacy Controls, especially where logging, auditability, and traceability must hold up during incident response or compliance review.
In practice, many security teams discover the architecture gap only after an investigation stalls because the raw events were kept, but the identity and timing context needed to explain them was not.
How It Works in Practice
The most durable designs separate ingestion, normalization, storage, and detection. High-priority events such as authentication failures, privilege changes, endpoint detections, cloud control-plane actions, and SOAR-triggering alerts should flow into the SIEM first, where detection logic can act on them in near real time. Broader telemetry, including DNS, proxy, application logs, data access records, and cloud audit trails, can land in the data lake for lower-cost retention and deeper correlation.
The key is to enrich data before it loses operational value. Identity, device, workload, and application metadata should be added as early as possible so that downstream analytics can answer basic questions without reconstructing context from multiple systems. This is especially important for NHI, service accounts, API keys, and agentic AI workloads, where the actor may not be a person but still represents an execution path with privilege. If the event record does not preserve subject, privilege, source, and session details, the lake becomes a warehouse of incomplete evidence rather than a forensic asset.
- Use the SIEM for correlation rules, alert triage, and response workflows.
- Use the lake for long retention, feature extraction, hunting, and model training support where appropriate.
- Define a common schema so records can move between SIEM, lake, and case management without rework.
- Apply retention, access control, and integrity protections consistently across both tiers.
Operationally, this means building a pipeline that normalizes events once and reuses the result everywhere, rather than creating separate pipelines for each consumer. It also means deciding which detections require immediate alerting and which can be batch-analysed later. Guidance from the CISA Incident Response Playbook is useful here because it reinforces the need for fast triage data and preserved evidence chains. These controls tend to break down in multi-cloud environments with inconsistent logging services and fragmented identity sources because correlation keys are not stable across platforms.
Common Variations and Edge Cases
Tighter retention and enrichment policies often increase storage and pipeline overhead, requiring organisations to balance investigative depth against cost and operational complexity. That tradeoff is especially visible when the SIEM is used as a compliance engine, a hunting platform, and a 24/7 alerting system all at once.
Some teams keep the SIEM intentionally narrow and push almost everything else into the lake. That can work when detection engineering maturity is high, but current guidance suggests it is risky if the team cannot guarantee low-latency access to the same records during an incident. Other teams route too much into the SIEM and then suppress alerts because licensing or performance constraints become unmanageable. The better approach is to define tiered telemetry by use case: critical security events for alerting, broad telemetry for analysis, and immutable copies for evidence preservation.
There is no universal standard for this yet, but best practice is evolving toward identity-centric observability. That matters for privileged sessions, ephemeral cloud workloads, and AI agents that invoke tools on behalf of a human or service principal. If the environment includes regulated workloads, map the design to NIST Cybersecurity Framework 2.0 for governance and detection outcomes, and use MITRE ATT&CK to test whether the SIEM retains enough fidelity for real adversary techniques. The model is weakest where log sources are ephemeral, user attribution is indirect, or sensitive data must be tokenised before storage.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK address the attack and risk surface, while NIST CSF 2.0 and NIST SP 800-53 Rev 5 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | DE.CM | Continuous monitoring underpins the SIEM side of the hybrid design. |
| MITRE ATT&CK | T1078 | Valid Accounts is a common technique that demands strong identity correlation. |
| NIST SP 800-53 Rev 5 | AU-2 | Audit event selection drives what must be captured for monitoring and forensics. |
Define which telemetry must be monitored continuously and send it to the SIEM for alerting and response.
Related resources from NHI Mgmt Group
- How should security teams decide what identity data belongs in a hybrid SIEM?
- How should security teams design taxonomy for sensitive data protection?
- How should security teams govern AI access to sensitive data across hybrid environments?
- How should security teams govern data lineage across hybrid and multi-cloud environments?
