Join our Newsletter — 33% off our NHI Course

Notifications
Clear all

Aggregation windows and event-driven detections: what changes for analysts?


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 20377
Topic starter  

TL;DR: Aggregation is not one operation, contrasting grouped summaries, running state, and event-preserving windowed detections across SQL, Kusto, Splunk, Cribl, and TQL, according to TENZIR. The practical shift is from memorising product-specific commands to controlling boundary, state, and output explicitly, which matters when detections must keep the triggering event and its context.

NHIMG editorial — based on content published by TENZIR: Aggregation is not one operation

By the numbers:

Questions worth separating out

Q: How should security teams use trailing windows in authentication detections?

A: Use trailing windows when the meaning of an event depends on the immediately preceding history, such as a successful login after repeated failures.

Q: Why do summary-only aggregations create blind spots in identity monitoring?

A: Summary-only aggregations can hide the original event, which is often the only record that contains the session, device, source address, and user agent needed for investigation.

Q: What do security teams get wrong about rolling event counts?

A: They often assume a rolling count is enough on its own, even when the alert also needs the event that crossed the threshold.

Practitioner guidance

  • Separate detection outputs from summary metrics Define detections so the triggering event is preserved with attached context, while aggregate-only reporting stays in a separate pipeline.
  • Use trailing windows for burst-based authentication logic Apply trailing windows to success-after-failure patterns, password spraying checks, and similar identity sequences where the decisive event is the last event in the series.
  • Explicitly define admission, contribution, and trigger rules Document which events enter retained history, which events count toward the aggregate, and which events cause evaluation.

What's in the full article

TENZIR's full article covers the operational detail this post intentionally leaves for the source:

  • Line-by-line comparisons of Splunk, Cribl Stream, and TQL query shapes for stateful detection logic
  • Worked examples showing how trailing windows preserve the successful login and its context
  • Operator-specific details on TQL window size, trigger, and summarize semantics
  • Migration considerations for teams translating existing search logic into pipeline-native detections

👉 Read TENZIR's analysis of stateful aggregation for event-driven detections →

Aggregation windows and event-driven detections: what changes for analysts?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 4 months ago
Posts: 19968
 

Event-preserving aggregation is a governance problem, not just a query syntax problem. Security teams often need the original login, token, or API event plus its preceding context, because the summary row alone is not actionable. When detection logic collapses the event too early, identity signals lose the session, device, and source details needed for triage. Practitioners should treat output shape as part of the control design, not a formatting choice.

A question worth separating out:

Q: How do teams keep stateful detections correct in parallel pipelines?

A: They need deterministic routing so all events for the same key, such as a user or service principal, are processed by the same state owner. Otherwise partial histories can split across workers and thresholds can fail silently. Correct keyed state is as much a reliability issue as a detection issue.

👉 Read our full editorial: Why trailing aggregation windows matter for event-driven detections



   
ReplyQuote
Share: