Join our Newsletter — 33% off our NHI Course

Notifications
Clear all

Security data lake storage at the edge: what should teams change?


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 18004
Topic starter  

TL;DR: Security data storage should be split across edge retention, centralized hot and cold tiers, and stream processing so teams can support debugging, long-term archive, and real-time analytics without forcing every event through the SIEM, according to Axoflow. The governance question is no longer whether to centralize all logs, but which data belongs at the edge, which requires correlation, and how to keep access and retention aligned with operational need.

NHIMG editorial — based on content published by Axoflow: Axoflow's Storage Strategy: Building the Security Data Layer

By the numbers:

Questions worth separating out

Q: How do teams decide whether to process telemetry at the edge or in the cloud?

A: Use data sensitivity, latency, bandwidth, and operational control as the deciding factors.

Q: Why do raw logs create problems for identity and access investigations?

A: Raw logs show that an action occurred, but they rarely show whether the actor was expected, whether the location was normal, or whether the source had a known threat reputation.

Q: What do teams get wrong about federated search and local log storage?

A: They often assume that distributed query capability removes the need for central governance.

Practitioner guidance

  • Define storage tiers by investigative value Classify logs into edge-only, hot, and cold categories based on how often they are queried, how long they must be retained, and whether they support identity investigations.
  • Apply least privilege to query surfaces Restrict who can run federated searches across distributed stores and who can access raw edge telemetry.
  • Review routing policy as a control Document which streams are enriched, aggregated, or forwarded at the edge, and verify that policy-based routing does not suppress evidence needed for incident response or compliance.

What's in the full article

Axoflow's full post covers the operational detail this article intentionally leaves for the source:

  • AxoStore query behaviour for short-window local retention and node-level troubleshooting
  • AxoLake hot-tier and cold-tier storage design choices for long-term security retention
  • AxoRouter policy logic for edge aggregation, enrichment, and forwarding decisions
  • Implementation context for teams deciding when to centralize logs versus keep them local

👉 Read Axoflow's analysis of security data layer storage at the edge →

Security data lake storage at the edge: what should teams change?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 3 months ago
Posts: 17593
 

Security data architecture is becoming a governance problem, not just an ingest problem. Once storage is split between edge nodes, hot tiers, and cold archive, teams are no longer managing a single telemetry plane. They are managing multiple access surfaces, multiple retention windows, and multiple ways for evidence to become unavailable or inconsistent. That makes governance, not raw capacity, the defining control issue.

A question worth separating out:

Q: How do organisations keep edge storage from weakening incident response?

A: Require that operational data with potential forensic value is promoted into a controlled archive, and verify that incident responders can retrieve it without relying on a single node or manual workaround. The key is preserving evidence path continuity before an incident forces the issue.

👉 Read our full editorial: Security data layer storage changes how teams balance edge and centralization



   
ReplyQuote
Share: