Streaming network detections turn flow telemetry into portable findings

At a glance

What this is: This is a technical guide to building streaming network detections from flow telemetry, showing how shared OCSF mappings let scan, beacon, exfiltration, and long-connection logic run across multiple sources.

Why it matters: It matters because SOC and security engineering teams need reusable detection logic that survives changes in collectors, sensors, and log formats without losing fidelity or breaking correlation.

👉 Read TENZIR's guide to streaming network detections from flow telemetry

Context

Flow telemetry can expose scanning, command-and-control cadence, and data exfiltration without payload inspection, but only if the detection layer treats records as a shared contract rather than a source-specific log format. That is the core governance problem in network detection engineering: collection changes constantly, while analytic logic must stay stable across exporters, sensors, and cloud logs.

The article also has an identity-adjacent angle, but it is indirect. It shows how security telemetry normalisation and detection portability depend on disciplined data semantics, the same kind of control discipline that identity teams apply when they standardise access signals across systems. For readers responsible for SOC, cloud, or GRC programmes, the practical issue is whether detections can remain trustworthy as telemetry sources multiply.

Key questions

Q: How should security teams design flow-based detections that work across different telemetry sources?

A: Start by normalising every source into a stable schema and then build detections against that schema, not the original log format. Keep field semantics, event time, and provenance consistent so the same logic can run across NetFlow, Zeek, Suricata, and cloud flow logs without rewriting the detector for each source.

Q: When do scan and beacon detections fail in practice?

A: Scan logic fails when fan-out is low but intent is hostile, while beacon logic fails when communication is irregular or hidden inside one long-lived session. Both can also fail if source fields are inconsistent, because timing and destination counts become unreliable. Pairing complementary detectors is the safer approach.

Q: How do security teams know whether flow telemetry detections are actually working?

A: Look for detections that are explainable, tunable, and reproducible across sources. If a rule only works for one sensor, or if threshold changes cannot be traced to representative traffic, the programme is too brittle. Good flow detections preserve provenance and emit findings that analysts can triage quickly.

Q: What should teams do when beacon detection misses long connections?

A: Add a companion duration-based control that flags sessions which never close long enough to produce repeated check-ins. That gives you visibility into tunnels and implants that evade cadence analysis by holding one connection open. Use it with outbound volume and destination context to reduce noise.

Technical breakdown

Why OCSF mapping matters for streaming detections

The article’s central mechanism is schema normalisation. NetFlow, Zeek, Suricata, and cloud flow logs can all feed the same detection logic only if they map into stable OCSF Network Activity fields such as src_endpoint, dst_endpoint, connection_info, traffic, and time. That allows the detector to ignore the original producer and work against a common event contract. This matters because detection engineering fails when logic is coupled to one sensor’s quirks or field names. OCSF does not create truth by itself, but it makes the truth machine-readable across sources.

Practical implication: Treat field mapping as part of detection quality, not just ingestion plumbing.

How scan fan-out detection separates probing from normal clients

Scan detection here uses breadth plus failure signals. A scanner touches many hosts or ports in a short period, usually with small flows and a high share of unanswered TCP attempts. The detector groups by source over a five-minute event-time window, counts distinct destinations and ports, and checks whether most TCP probes never received an ACK. That combination reduces false positives from busy but legitimate clients that still complete their connections. The design is deliberately explicit: thresholds stay visible and tunable rather than hidden inside a composite score.

Practical implication: Tune fan-out, packet-count, and unanswered-share thresholds against representative traffic before operationalising the detector.

What beacon cadence and long connections reveal about command-and-control

Beaconing is a timing problem, not just a volume problem. The detector measures inter-arrival intervals and flow sizes for a fixed source-destination-port tuple, then uses median, MAD, and skewness to identify regular check-ins with small payloads. That works because implants often sleep, wake, send a small request, and repeat. The article also notes the blind spot: a single long-lived connection can hide the repeated-flow pattern entirely, which is why a separate long-connection detector is needed. Together, these patterns cover both periodic beacons and persistent tunnels.

Practical implication: Pair cadence analysis with long-connection reporting so one sustained session does not evade beacon logic.

Threat narrative

Attacker objective: The attacker wants to discover exposed targets, maintain covert control, and move data out of the environment while avoiding payload-based detection.

1. Entry occurs when an attacker establishes scanning or reconnaissance traffic that fans out across many hosts or ports without meaningful application interaction.
2. Escalation appears when repeated outbound check-ins or sustained tunnels create command-and-control channels that support tasking, staging, or follow-on access.
3. Impact follows when the same telemetry pattern is used to move data out of the environment through exfiltration bursts or to maintain covert control for later actions.

NHI Mgmt Group analysis

Portable detection logic is now a governance issue, not just an engineering convenience. When detectors operate against a shared OCSF contract, the control problem shifts from parsing logs to validating semantics. That is a stronger foundation for SOC portability, but only if teams treat field mapping, provenance, and event-time handling as part of the control surface. The practitioner conclusion is simple: standardise the detection interface before you optimise the analytics.

Cadence-based beacon detection is most useful when paired with a second control for persistent sessions. The article correctly identifies the structural blind spot in beacon hunting: one long connection can hide repeated check-ins. That is a useful lesson for SOC design because it separates periodicity from persistence, two different forms of hostile communication. The practitioner implication is to build complementary detections instead of assuming one statistical pattern covers all command-and-control behaviour.

Detection engineering debt is the hidden risk this article exposes. The more detection logic depends on source-specific fields, the more fragile the programme becomes as telemetry expands across NetFlow, Zeek, Suricata, and cloud logs. Standardised mapping reduces that debt by making detections reusable, testable, and easier to route into downstream findings. The practitioner conclusion is to measure how much of your alerting still depends on one-off parsing logic.

Network telemetry can support zero-trust visibility, but not zero-trust by itself. The article shows how enriched flow data can identify suspicious behaviour, yet it does not establish identity, intent, or policy enforcement. That distinction matters for broader security architecture: detection visibility and access control are different layers. The practitioner conclusion is to use flow analytics as an evidence source inside a larger trust model, not as a substitute for it.

What this signals

Security teams should expect more value from detection pipelines that separate schema mapping from analytic logic. That design makes it easier to swap collectors, preserve provenance, and scale into cloud flow telemetry without rewriting every rule.

Detection engineering debt: as telemetry sources multiply, the real risk is not lack of alerts but lack of portability. Programmes that cannot reuse detection logic across NetFlow, Zeek, and Suricata will spend more time maintaining parsers than improving coverage.

The strongest near-term improvement is operational discipline around event-time windows, enrichment, and complementary detectors. That combination supports higher-fidelity hunting without forcing payload inspection into every decision.

For practitioners

• Standardise flow fields before writing detections Map every supported source into a common OCSF Network Activity contract and validate that src_endpoint, dst_endpoint, traffic, connection_info, and time behave consistently across exporters.
• Separate scan, beacon, and exfiltration logic Keep scan fan-out, cadence analysis, and outbound volume burst detection as independent rules so each threshold can be tuned, tested, and explained on its own.
• Add a long-connection detector next to beacon hunting Cover the blind spot where a single tunnel stays open and never produces repeated check-ins, using duration-based logic alongside cadence analysis.
• Preserve provenance through enrichment and routing Carry source metadata forward so hunters can distinguish NetFlow, Zeek, Suricata, and cloud flow records after normalisation and still trace evidence back to the original telemetry source.

Key takeaways

• Flow telemetry can surface scanning, beaconing, and exfiltration without payload inspection when detections are built on a shared schema.
• The article’s most important operational lesson is that cadence analysis needs a companion control for long-lived connections.
• Teams that standardise mappings and preserve provenance will be able to reuse detection logic across collectors instead of rebuilding it for each source.

Key terms

• OCSF Network Activity: A normalised event class for network telemetry that lets different producers share a common structure. It preserves key connection fields, metadata, and provenance so analytics can run across multiple sources without rewriting the detection logic for each sensor.
• Beacon Cadence: A repeating timing pattern in outbound network traffic that can indicate command-and-control behaviour. Analysts measure the spacing and size of flows to find small, regular check-ins, then compare them against normal client behaviour and expected operational noise.
• Detection Finding: A structured alert object that carries analytic results, evidence, and context into downstream systems. Unlike a raw log entry, it is designed to be routed, correlated, and acted on by other tools while preserving enough metadata for investigation and audit.

What's in the full article

TENZIR's full article covers the operational detail this post intentionally leaves for the source:

• The complete TQL operator patterns for scan fan-out, beacon cadence, outbound burst, and long-connection detection
• Threshold arguments and tuning guidance for event-time windows, dispersion limits, and unanswered TCP shares
• Example OCSF finding envelopes that show how detection results are emitted and correlated
• The full mapping and enrichment approach for keeping provenance intact across NetFlow, Zeek, and Suricata sources

👉 TENZIR's full article shows the detector logic, tuning thresholds, and OCSF finding structure in detail

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, secrets management, workload identity, and agentic AI identity. It is designed for practitioners who need a structured identity security foundation across modern security programmes.