Join our Newsletter — 33% off our NHI Course

Notifications
Clear all

Flow telemetry detection in streaming pipelines: what it means for SOC teams


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 17031
Topic starter  

TL;DR: Scan fan-out, beacon cadence, exfiltration bursts, and long connections can be detected from flow records alone by mapping NetFlow, Zeek, and Suricata into a shared OCSF Network Activity stream, according to TENZIR. The key shift is that detection logic becomes portable and tunable, but only if field semantics stay stable and enrichment preserves trust boundaries.

NHIMG editorial — based on content published by TENZIR: streaming network detections from flow telemetry

Questions worth separating out

Q: How should security teams design flow-based detections that work across different telemetry sources?

A: Start by normalising every source into a stable schema and then build detections against that schema, not the original log format.

Q: When do scan and beacon detections fail in practice?

A: Scan logic fails when fan-out is low but intent is hostile, while beacon logic fails when communication is irregular or hidden inside one long-lived session.

Q: How do security teams know whether flow telemetry detections are actually working?

A: Look for detections that are explainable, tunable, and reproducible across sources.

Practitioner guidance

  • Standardise flow fields before writing detections Map every supported source into a common OCSF Network Activity contract and validate that src_endpoint, dst_endpoint, traffic, connection_info, and time behave consistently across exporters.
  • Separate scan, beacon, and exfiltration logic Keep scan fan-out, cadence analysis, and outbound volume burst detection as independent rules so each threshold can be tuned, tested, and explained on its own.
  • Add a long-connection detector next to beacon hunting Cover the blind spot where a single tunnel stays open and never produces repeated check-ins, using duration-based logic alongside cadence analysis.

What's in the full article

TENZIR's full article covers the operational detail this post intentionally leaves for the source:

  • The complete TQL operator patterns for scan fan-out, beacon cadence, outbound burst, and long-connection detection
  • Threshold arguments and tuning guidance for event-time windows, dispersion limits, and unanswered TCP shares
  • Example OCSF finding envelopes that show how detection results are emitted and correlated
  • The full mapping and enrichment approach for keeping provenance intact across NetFlow, Zeek, and Suricata sources

👉 Read TENZIR's guide to streaming network detections from flow telemetry →

Flow telemetry detection in streaming pipelines: what it means for SOC teams?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 3 months ago
Posts: 16386
 

Portable detection logic is now a governance issue, not just an engineering convenience. When detectors operate against a shared OCSF contract, the control problem shifts from parsing logs to validating semantics. That is a stronger foundation for SOC portability, but only if teams treat field mapping, provenance, and event-time handling as part of the control surface. The practitioner conclusion is simple: standardise the detection interface before you optimise the analytics.

A question worth separating out:

Q: What should teams do when beacon detection misses long connections?

A: Add a companion duration-based control that flags sessions which never close long enough to produce repeated check-ins. That gives you visibility into tunnels and implants that evade cadence analysis by holding one connection open. Use it with outbound volume and destination context to reduce noise.

👉 Read our full editorial: Streaming network detections turn flow telemetry into portable findings



   
ReplyQuote
Share: