Notifications
Clear all
Topic starter
14/09/2026 10:09 am
TL;DR: Full-fidelity log retention is being squeezed by rising ingest prices, growing telemetry volumes, and expensive retrieval, according to Anomali’s analysis of Microsoft, Dynatrace, and Splunk pricing and survey data. The practical shift is from hot-index dependence to decoupled storage and query, because long retention only works when searchability no longer inherits SIEM economics.
NHIMG editorial — based on content published by Anomali: Full-Fidelity at Cold-Storage Cost: The Retention Economics
By the numbers:
- Dynatrace's 2026 State of Log Management report found log and telemetry volume rose 93 percent on average over the prior year.
- 150 percent, organisations saw log and telemetry growth above 150 percent, according to Dynatrace's 2026 State of Log Management report.
Questions worth separating out
Q: How should security teams design log retention when investigation history must stay searchable?
A: Design retention in two layers.
Q: Why does expensive retrieval undermine retrospective identity investigations?
A: Because investigators do not just need the data to exist.
Q: What are the signs that a retention model is failing security operations?
A: The clearest signs are shortened hot windows, delayed historical hunts, and investigators avoiding broad retrospective searches because the process is too slow or costly.
Practitioner guidance
- Separate hot detection from long-term retention Keep recent, high-value events in a fast analytics tier, but move older telemetry into low-cost object storage with a governed query layer so retention depth is no longer bounded by ingest pricing.
- Normalise identity telemetry before archiving Map Windows, SaaS, and cloud authentication events into a shared schema such as OCSF so investigators can query one model across identity sources instead of translating every source at hunt time.
- Define an investigation retention window by threat model Set retention periods based on the longest realistic access, identity abuse, or insider investigation window your team must support, then validate that the stored data remains searchable throughout that period.
What's in the full article
Anomali's full article covers the operational detail this post intentionally leaves for the source:
- Pricing examples showing how different retention tiers change the economics of keeping telemetry searchable over time
- Worked cost comparisons for interactive retention, archive, and object-storage query models at SOC-scale volumes
- A practical explanation of how schema normalisation makes year-old identity data queryable across Windows, SaaS, and cloud sources
- The specific assumptions behind Anomali's cost-reduction claim in a large financial-institution deployment
👉 Read Anomali's analysis of full-fidelity retention economics →
SOC data retention costs: what teams are missing in practice?
Explore further
14/09/2026 10:10 am
Retention economics is now an identity governance problem, not just a storage problem. When organisations cannot afford to keep authentication history searchable, they lose the evidence needed to answer who accessed what, from where, and by which identity. That weakens human IAM review, NHI forensics, and account abuse investigations alike. The practical conclusion is that retention architecture now belongs in identity governance conversations, not only in SOC platform planning.
A question worth separating out:
Q: When should organisations move from SIEM-centric retention to decoupled data-lake retention?
A: When telemetry volume, query cost, and retention requirements no longer fit inside a single ingest-and-index model. If the team needs months of identity history for investigations, but only a small portion is queried routinely, decoupled retention becomes the more workable architecture.
👉 Read our full editorial: Retention economics expose the real cost of full-fidelity SOC data
