Security teams should normalise telemetry as close to collection as practical, using a shared schema so routing, retention, and detection all see the same event structure. The goal is not cosmetic consistency. It is preserving meaning across identity, cloud, endpoint, and network sources so analytics can operate without constant translation overhead.
What Normalisation Should Accomplish for SIEM and Lakehouse Pipelines
Normalisation should turn raw telemetry into a stable analytical contract, not just a cleaner-looking record. In practice that means choosing the same event fields, data types, timestamps, and identity markers early enough that downstream consumers do not each invent their own translation layer. The result is better correlation, simpler retention policy, and fewer detection gaps caused by schema drift.
The key design choice is where to pay the translation cost. If normalisation happens only at query time, every dashboard, detection rule, and lakehouse job has to relearn the same source quirks. If it happens close to collection, routing and storage can preserve a consistent structure while still keeping the raw event when forensic fidelity matters.
For teams building mixed estates, the practical target is a shared schema with controlled extensions. Core fields should be consistent across identity, cloud, endpoint, and network telemetry, while source-specific attributes can remain as nested or optional data. That approach keeps the high-value fields usable for detection and hunting without flattening away the details analysts need later.
How to Normalise Without Breaking Fidelity
The safest pattern is to separate semantic normalisation from destructive transformation. Semantic normalisation aligns concepts such as actor, target, action, outcome, and session across sources. Destructive transformation, such as dropping fields or over-aggregating events, should be avoided unless the use case has been explicitly accepted by detection and forensic owners.
Event time is usually the first field to standardise, followed by entity identifiers and outcome status. Teams should be especially careful with user IDs, hostnames, cloud resource IDs, and API-related identifiers because those values often drive correlation across products. If the same entity cannot be joined reliably, the lakehouse becomes a storage layer rather than an analytics layer.
Normalisation should also preserve provenance. Analysts need to know which collector, parser, or enrichment step produced each field so they can distinguish original source content from derived content. This matters when a detection fires on an enriched attribute but the investigator later needs to validate the original telemetry path.
Why Normalisation Fails in Real Environments
Most failures come from teams optimising for ingestion speed or dashboard convenience instead of analytical consistency. A common mistake is allowing each security product to emit its own near-duplicate schema, then trying to reconcile everything downstream. Another is over-normalising at the edge, which can remove source nuance that is later needed to explain suspicious behaviour or prove an event sequence.
Lakehouse architectures make this problem more visible because they invite multiple consumers with different expectations. If the same event lands in the SIEM, a hunting workspace, and a data science notebook with different field conventions, the organisation will eventually dispute which version is authoritative. That dispute is usually a schema-governance problem, not a tooling problem.
Normalisation also affects identity-heavy telemetry. Authentication, privilege, and session events need consistent subject and actor fields so correlation does not break when the source changes from cloud control plane to endpoint agent to directory service. For teams handling those joins, this Sumo Logic breach case is a reminder that log platforms and their access paths are themselves sensitive operational dependencies.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0 and NIST SP 800-53 Rev 5 set the technical controls, while ISO/IEC 27001:2022 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.PO-01 — Policy | Telemetry normalisation needs a governed data and logging policy across tools. |
| Recommendation — Define a telemetry schema policy that standardises fields across SIEM and lakehouse pipelines. | ||
| NIST SP 800-53 Rev 5 | AU-3 — Content of Audit Records | Normalisation depends on consistent audit record content for correlation and detection. |
| AU-12 — Audit Record Generation | Collection-stage normalisation depends on generating the right event fields at source. | |
| Recommendation — Standardise audit record content so events remain comparable across sources and platforms. Generate audit records with the fields needed for downstream normalisation and analysis. | ||
| ISO/IEC 27001:2022 | A.8.15 — Logging | Logging controls require consistent event capture and handling across systems. |
| A.8.16 — Monitoring activities | Normalised telemetry improves the monitoring and correlation expected by this control. | |
| Recommendation — Align logging formats and retention so telemetry stays usable across SIEM and lakehouse workflows. Use a shared telemetry schema to support reliable monitoring and alert correlation. | ||
Practitioner Guidance
What to prioritise: Define a small set of canonical fields first, then map each source into those fields before adding enrichment. Prioritise the fields your detections and investigations actually join on, not the fields that are easiest to extract.
What to verify: Check that one event can be traced from collection to SIEM to lakehouse without changing its core meaning. If a record cannot survive that path with stable actor, time, action, and target semantics, the normalisation design is too brittle.
Common mistake: Treating normalisation as a parser exercise. The real decision is governance over meaning, because inconsistent semantics create silent detection failures even when ingestion is technically successful.
What good looks like: Detection engineers, hunters, and data consumers can query the same event family without building one-off mappings for every source. At that point the SIEM and lakehouse are reinforcing the same analytical model instead of competing schema versions.
Practitioner takeaway: Normalise early enough to preserve shared meaning, but not so aggressively that you erase the evidence needed for investigation and reconstruction.
Related resources from NHI Mgmt Group
- How should security teams decide whether JIT access is safe for non-human identities?
- How should security teams decide which telemetry belongs in the SIEM?
- How should security teams implement telemetry pipelines before the SIEM?
- How should security teams implement decoupled telemetry architectures without losing visibility?