Join our Newsletter — 33% off our NHI Course

How did Parquet make long-term log storage cheaper than SIEMs?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 2 years ago
Posts: 19717
Topic starter  

TL;DR: Parquet-backed log retention is the practical answer Axoflow gives for years of security data that outgrow SIEM economics. Object storage and open formats keep archives queryable while separating hot detection from cold retention.

Editorial analysis by NHI Mgmt Group, based on content published by Axoflow: “Long-Term Log Storage Without SIEM Costs: The Axoflow Storage Layer”.

Key questions

Q: Why do long-retention archives need open data formats?

A: Open formats preserve access after the original toolchain changes.

Q: How should security teams separate analytics from retention storage?

A: Use the SIEM for short-lived analytics and investigations, then place historical evidence in object storage built for low-cost retention.

Q: What breaks when teams use a SIEM as long-term log storage?

A: The storage model becomes misaligned with the workload.

Practitioner guidance

  • Separate hot analytics from cold retention Keep SIEMs focused on detection and short-retention analytics, then move multi-year retention into cheaper object storage designed for archive economics.
  • Adopt open, self-describing storage formats Store retained logs in formats such as Parquet so the archive remains readable and queryable even if the original platform is no longer available.
  • Validate rehydration after platform loss Test whether teams can still query historical logs if the SIEM, local indexes, or original query service are unavailable.

Bottom line: Long-term log storage fails when organisations force retention workloads into SIEMs designed for analytics and alerting.

Glossary terms: Log Retention, Open File Format, Columnar Database, Object Storage

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →


This topic was modified 1 day ago by NHI Mgmt Group

   
Quote
(@mr-nhi)
Member Moderator
Joined: 6 months ago
Posts: 20207
 

Open-format archive design is becoming a governance requirement, not a storage preference. When retention stretches from months to years, the question is no longer only where logs sit, but whether they remain usable after tooling changes, mergers, platform exits, or cloud shifts. A self-describing format reduces operational dependency and strengthens evidence continuity. For identity and security programmes, that means retention strategy now belongs in architecture governance, not just infrastructure procurement.

A question worth separating out:

Q: What happens if retained logs cannot be rehydrated later?

A: Investigations and audits become dependent on the original platform remaining available and intact. If the archive cannot be reopened independently, the organisation risks losing the ability to reconstruct access history, privilege use, or incident timelines when they matter most.

👉 Read our full editorial: Long-term log storage beyond SIEM costs needs open formats



   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.