Notifications
Clear all
Topic starter
18/08/2026 2:43 pm
TL;DR: A rolling exfiltration detector becomes memory-bound when it tries to learn long-horizon baselines inside the stream, according to TENZIR. The operational lesson is that detection quality improves when state lives in the lake, not the live window, and TENZIR argues for separating learning from judging by publishing per-asset statistics from durable storage.
NHIMG editorial — based on content published by TENZIR: the rolling exfiltration detector and durable baseline pipeline
Questions worth separating out
Q: How should security teams implement long-horizon anomaly detection without bloating streaming state?
A: Keep the live detector narrow and push historical learning into a scheduled job that reads durable telemetry, computes per-entity baselines, and writes the result to a lookup table or materialized view.
Q: Why do entity-specific baselines work better than global thresholds for egress monitoring?
A: Because normal behaviour varies sharply by asset, account, and workload.
Q: What breaks when training data includes malicious behaviour?
A: The baseline can absorb the attack path as normal, which weakens or removes the deviation signal entirely.
Practitioner guidance
- Separate learning from detection Run scheduled baseline computation in a durable store and keep the streaming detector limited to live scoring and enrichment.
- Baseline by asset, account, or workload Group behaviour by the specific entity you are judging, not by the whole environment.
- Add freshness checks to baseline use Refuse or downgrade baselines when updated_at is stale, so a silent learner failure becomes visible instead of quietly degrading detection quality.
What's in the full article
TENZIR's full article covers the operational detail this post intentionally leaves for the source:
- Runnable TQL examples for daily per-asset baseline calculation and lookup-table publication
- ClickHouse-backed query patterns for retaining weeks of network activity without expanding detector memory
- Alternative baseline placements, including materialized views and hybrid lake plus detector models
- Handling null baselines, freshness checks, and seasonality in the scoring pipeline
👉 Read TENZIR's analysis of durable baselines for network egress detection →
Network egress baselines: how should teams govern long-horizon detection?
Explore further
18/08/2026 3:26 pm
Long-horizon detection is a state-governance problem, not just a thresholding problem. Once a detector has to remember weeks of entity behaviour, it starts behaving like a data store with alert logic attached. That creates fragility around restarts, refresh cadence, and state ownership. For practitioners, the right question is whether the control owns history cleanly or smuggles it into runtime memory.
A question worth separating out:
Q: Who should own the baseline refresh logic in a detection pipeline?
A: The team that owns detection operations should own the refresh contract, even if another system computes the baseline. Clear ownership matters because the detector needs a dependable freshness signal, defined failure handling, and a stable enrichment schema. Without that governance, the control becomes hard to audit and easy to break during maintenance.
👉 Read our full editorial: Scheduled baselines for network egress detection need durable state
