Network egress baselines: how should teams govern long-horizon detection?

TL;DR: A rolling exfiltration detector becomes memory-bound when it tries to learn long-horizon baselines inside the stream, according to TENZIR. The operational lesson is that detection quality improves when state lives in the lake, not the live window, and TENZIR argues for separating learning from judging by publishing per-asset statistics from durable storage.

NHIMG editorial — based on content published by TENZIR: the rolling exfiltration detector and durable baseline pipeline

Questions worth separating out

Q: How should security teams implement long-horizon anomaly detection without bloating streaming state?

A: Keep the live detector narrow and push historical learning into a scheduled job that reads durable telemetry, computes per-entity baselines, and writes the result to a lookup table or materialized view.

Q: Why do entity-specific baselines work better than global thresholds for egress monitoring?

A: Because normal behaviour varies sharply by asset, account, and workload.

Q: What breaks when training data includes malicious behaviour?

A: The baseline can absorb the attack path as normal, which weakens or removes the deviation signal entirely.

Practitioner guidance

• Separate learning from detection Run scheduled baseline computation in a durable store and keep the streaming detector limited to live scoring and enrichment.
• Baseline by asset, account, or workload Group behaviour by the specific entity you are judging, not by the whole environment.
• Add freshness checks to baseline use Refuse or downgrade baselines when updated_at is stale, so a silent learner failure becomes visible instead of quietly degrading detection quality.

What's in the full article

TENZIR's full article covers the operational detail this post intentionally leaves for the source:

• Runnable TQL examples for daily per-asset baseline calculation and lookup-table publication
• ClickHouse-backed query patterns for retaining weeks of network activity without expanding detector memory
• Alternative baseline placements, including materialized views and hybrid lake plus detector models
• Handling null baselines, freshness checks, and seasonality in the scoring pipeline

👉 Read TENZIR's analysis of durable baselines for network egress detection →

Network egress baselines: how should teams govern long-horizon detection?

Explore further

View Full Forum →  |  NHI Foundation Course →