Most teams need both, but for different purposes. Normalized events support routine detection and low-cost analytics, while raw telemetry provides forensic depth when investigations require reconstruction. The key is to decide explicitly which data stays in expensive systems and which can be moved to lower-cost storage without reducing response capability.
Why This Matters for Security Teams
Choosing between raw telemetry and normalized events is really a question of investigative fidelity, retention cost, and detection speed. Normalized events are easier to query, compare, and route into standard detections, but they can strip away fields that matter during incident reconstruction. Raw telemetry preserves the original context, which is often essential for proving sequence, spotting evasion, and validating whether an alert reflects true malicious behaviour or a parsing artifact.
The operational risk is not just storage spend. If the team normalizes too early, it may lose details needed for threat hunting, fraud analysis, or post-incident scoping. If it keeps everything raw forever, the result is often an expensive log archive with weak searchability and unclear retention rules. The right balance should be tied to the control objectives in NIST Cybersecurity Framework 2.0, especially around detection, logging, and response maturity. In practice, many security teams discover they needed the raw record only after a containment decision has already been made from incomplete normalized data.
How It Works in Practice
Most mature environments use a tiered logging model. Ingestion pipelines first capture raw telemetry from sources such as endpoints, identity providers, cloud control planes, SaaS audit logs, and network tools. A second pipeline normalizes selected fields into a common schema so detections, dashboards, and correlation rules can operate efficiently. That separation lets teams keep a smaller, usable security data layer in hot storage while retaining raw records for replay, evidentiary review, and parser validation.
A practical design usually starts with retention classes:
- Hot normalized events for daily detection and SOC triage.
- Warm raw telemetry for investigations, tuning, and reconstruction.
- Cold archive for longer legal, regulatory, or internal retention needs.
This model works best when the normalization process is deterministic and versioned, because analysts need to know how a field was transformed and whether a parser change altered a detection path. It is also important to preserve timestamps, source identifiers, and integrity metadata so chain-of-events analysis remains credible. Guidance from the CISA insider threat mitigation resources and the logging expectations in CIS Controls both reinforce the value of retaining enough context to investigate misuse, not just trigger alerts. Where identity is central, raw logs may also be needed to trace session hijacking, token abuse, or privileged access activity across systems.
Teams should also define which raw sources are mandatory. For example, cloud API audit logs, authentication events, EDR telemetry, and administrative actions often deserve longer retention than noisy application diagnostics. The key is to align retention with the investigation questions the organisation expects to answer. These controls tend to break down in high-ingest environments with weak schema governance because normalization drift quickly makes the “standard” event layer less trustworthy than the raw source.
Common Variations and Edge Cases
Tighter raw retention often increases storage and privacy overhead, requiring organisations to balance forensic readiness against cost, access control, and data minimisation. That tradeoff becomes more pronounced when logs contain personal data, secrets, or regulated content.
There is no universal standard for how long raw telemetry must be kept, so current guidance suggests using risk-based retention by data class rather than one fixed rule. In privacy-sensitive environments, teams often keep raw data only for the systems most likely to require reconstruction, while normalized events support broader monitoring. For agentic AI or automated workflows, raw telemetry can also preserve prompt, tool, and execution context that normalized summaries may omit, which matters when investigating unsafe agent actions or policy violations.
Edge cases include ultra-high-volume environments, ephemeral cloud workloads, and distributed SaaS estates where raw retention across every source is impractical. In those cases, best practice is evolving toward selective raw capture, lossless compression, and strong metadata tagging so analysts can retrieve the right evidence quickly. If the question is not explicitly answered by a retention policy, a data minimisation review should decide what is discarded, what is summarized, and what must remain reconstructable for incident response and audit purposes.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST CSF 2.0 and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | DE.CM | Logging and monitoring depend on retaining enough telemetry to detect and investigate anomalies. |
| MITRE ATT&CK | T1070 | Adversaries often delete or tamper with logs, making retention depth important for detection and evidence. |
| OWASP Non-Human Identity Top 10 | Raw telemetry can expose token, session, and privileged identity misuse across non-human identities. | |
| NIST AI RMF | AI and agentic workflows need traceable execution context to explain behaviour and failures. |
Keep detectable context in logs so security monitoring can identify suspicious activity and support response.
Related resources from NHI Mgmt Group
- How should security teams decide which telemetry sources to retain in XDR programmes?
- How should IAM teams use customer events to assess governance maturity?
- How should teams use login telemetry to improve both security and customer experience?
- How should security teams use browser telemetry in identity risk management?
