Join our Newsletter — 33% off our NHI Course
Home› Guides› NetFlow and IPFIX Flow Telemetry Guide: Collecting and…
Guide Network Security

NetFlow and IPFIX Flow Telemetry Guide: Collecting and Using Flow Data for Detection

PDF for newsletter subscribers · free to read online

← All guides
By NHI Mgmt Group, edited by Lalit Choda Updated 27 October 2026 13 min read
On this page

Full packet capture does not scale to every link, but without network records a SOC cannot see lateral movement, beaconing or bulk exfiltration that never touches an endpoint agent. Flow telemetry fills that gap: routers, switches, firewalls and cloud platforms summarise each conversation (who talked to whom, on which ports, how much, when) and export the summaries to a collector. NetFlow version 9 and its IETF successor IPFIX are the main standards, with sFlow and cloud flow logs as close relatives. This guide explains the formats, which fields matter for security, how to design collection, how to secure the export path, and the detections and metrics that make flow data worth keeping. It is for network security engineers, SOC and detection engineers. For packet capture and general network monitoring, see the Network Monitoring and Packet Capture Guide.

Key takeaways

  • IPFIX is the standard; NetFlow v9 is its parent. RFC 7011 (IPFIX, STD 77, September 2013) is an Internet Standard with message version 10; RFC 3954 (NetFlow v9, October 2004) is an Informational description of Cisco's format.
  • Flows are metadata, not content. A flow record holds addresses, ports, protocol, byte and packet counts, timestamps and TCP flags, which is enough for many detections and cheap to keep for months.
  • Templates make the format flexible and fragile. Both formats describe records with templates; a collector that loses or never receives a template cannot decode the data.
  • Placement and sampling decide what you see. Unsampled flows at internet edges, data centre cores and cloud VPCs give the most security value; heavy sampling hides small, slow flows such as beacons.
  • The export path is an attack surface. Flow export usually runs over UDP without authentication, so isolate it on a management network or protect it with TLS or DTLS where supported.

Flow formats compared

FormatStatus and sourceStructureNotes for security teams
NetFlow v5Cisco legacy formatFixed record layoutIPv4 only and no extensibility; still found on older devices
NetFlow v9RFC 3954, Informational, October 2004; not a candidate for Internet StandardTemplate FlowSets (FlowSet ID 0) describe data records; Template IDs 256 to 65535; version field 9No fixed port in the RFC; UDP usual; templates expire at the collector unless refreshed
IPFIXRFC 7011, Internet Standard (STD 77), September 2013, obsoletes RFC 5101Template Sets (Set ID 2), Options Template Sets (Set ID 3), Data Sets 256 and above; version field 10Information Elements registered by IANA; supports variable-length and enterprise-specific fields
sFlowIndustry specification (sflow.org)Sampling of packets on switch and router interfacesDesigned for high-speed links; always sampled, so less suited to spotting single small flows
Cloud flow logs (for example AWS VPC Flow Logs)Provider documentationProvider-defined fields, for example the AWS version 2 default formatAggregated over an interval; delivery is best effort and delayed by minutes

RFC 3954 notes that NetFlow v9 was submitted as a basis for IETF work, and the IPFIX Working Group built IPFIX on it, so the two share concepts and many field numbers. Most modern collectors accept both.

How flow export works

The Cisco Flexible NetFlow model is a useful way to understand any exporter:

  • Flow record: a set of key fields (with match statements) that define what makes a new flow, and non-key fields (with collect statements) such as byte and packet counters.
  • Flow monitor: applied to an interface; holds a cache built from the record, ages out entries using active and inactive timeouts, and exports aged-out entries.
  • Flow exporter: sends cache contents to one or more collectors; one exporter can serve several monitors and a monitor can use several exporters.
  • Sampler: analyses only a subset of packets to reduce device load, at the cost of accuracy.

On the collector side, RFC 7011 requires a collecting process to keep template information for the duration of each transport session and not carry templates over between sessions. Over UDP there is no session, so templates are resent periodically and a collector may need to buffer data records that arrive before their template. Template withdrawal does not apply over UDP. These rules explain the common symptom of "missing data for the first few minutes" after a collector restart.

Fields that matter for security

IPFIX Information Elements are registered by IANA. The core set used by most detections:

Information ElementIANA IDSecurity use
sourceIPv4Address / destinationIPv4Address8 / 12Who talked to whom; matching against threat intelligence and asset inventory
sourceIPv6Address / destinationIPv6Address27 / 28The same for IPv6, often missing from older templates
sourceTransportPort / destinationTransportPort7 / 11Service identification; non-standard ports; scanning patterns
protocolIdentifier4TCP, UDP, ICMP and others; unusual protocols for tunnelling
octetDeltaCount / packetDeltaCount1 / 2Volume for exfiltration and denial of service detection
tcpControlBits6SYN-only flows for scans; RST patterns for rejected connections
ingressInterface / egressInterface10 / 14Direction and path through the device
ipClassOfService5Traffic class; occasionally abused for covert signalling
applicationId95Application classification where the exporter supports it
flowEndReason136Why the flow was exported (idle timeout, active timeout, end detected), which affects how long flows are stitched

In AWS, the default VPC Flow Logs version 2 format is version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status and cannot be changed. A custom format adds fields such as pkt-srcaddr and pkt-dstaddr (the original addresses behind a NAT gateway or EKS pod), tcp-flags (FIN 1, SYN 2, RST 4, SYN-ACK 18, OR-ed across the interval), flow-direction, traffic-path, vpc-id, instance-id and reject-reason. The default maximum aggregation interval is 600 seconds with a 60 second option, and AWS states delivery typically takes about 5 minutes to CloudWatch Logs and about 10 minutes to S3. For cloud log design see the Cloud Logging and Audit Trail Guide.

Designing collection

  • Where to export: internet edge (all ingress and egress), data centre and campus cores, links between security zones, VPN concentrators, and every cloud VPC or virtual network. Zone boundaries matter most because lateral movement crosses them. See the Network Segmentation Guide.
  • Sampling: use unsampled flow where devices can sustain it, especially at edges and on server segments. If sampling is unavoidable on very high-speed links, record the rate so analytics can scale counts, and accept that low-volume flows will be missed.
  • Timeouts: shorter active timeouts give fresher data for long-lived connections (useful for detecting persistent tunnels) at the cost of more records. Check your platform's defaults rather than assuming them.
  • Bidirectional records: decide whether the collector stitches the two directions of a conversation; detections such as ratio of bytes out to bytes in depend on it.
  • Enrichment: join flows with DHCP, asset inventory, identity (which user or workload owned an IP at that time), DNS logs and threat intelligence. Raw IP pairs are hard to act on.
  • Retention: flows are compact compared with packets, so organisations commonly keep them far longer than packet captures; set a period that covers your typical dwell time and investigation needs. See the Log Management and Retention Guide.
OptionBest forTrade-off
Flow onlyBroad coverage, long retention, cloud and branch sitesNo payload; limited application detail
Flow plus targeted packet captureMost enterprises: flow everywhere, packets at the edge and critical segmentsTwo pipelines to run
NDR sensor generating rich metadataDeep protocol analysis (for example TLS and DNS details) on key linksSensor cost and placement limits; see the NDR Guide

Small organisations: enable flow export on the internet firewall and cloud flow logs, send both to the SIEM, and start with a handful of detections. Large organisations: run a dedicated collector tier close to exporters, normalise NetFlow, IPFIX, sFlow and cloud logs to one schema, and feed both the SIEM and an analytics store sized for months of data.

Implementation: a phased rollout

  1. Inventory exporters. List devices and cloud accounts that can export flow, the formats they support (v5, v9, IPFIX, sFlow) and whether they support IPv6 fields.
  2. Build the collector path. Place collectors on a management network, restrict inbound export traffic by source address with ACLs, and size storage from a sample of exported volume.
  3. Pilot one boundary. Enable export on the internet edge with a full template (addresses, ports, protocol, counts, TCP flags, interfaces, timestamps) and validate against firewall logs for the same period.
  4. Expand by zone. Add data centre cores, zone boundaries and cloud VPCs. In AWS, use a custom format with pkt-srcaddr, pkt-dstaddr and tcp-flags and the 60 second interval where cost allows.
  5. Add enrichment and detections. Join with asset, identity and DNS data, then deploy detections in alert-only mode and tune before paging anyone.
  6. Test continuity. Restart a collector and confirm templates are reacquired; simulate an exporter failure and confirm the "no data" alert fires. Rollback for a device is removing the monitor from the interface, which stops export without affecting forwarding.

Securing the export path

Flow data reveals the whole shape of the network, and the export protocol is usually unauthenticated UDP, so an attacker on the path can read it and anyone who can reach the collector can inject false records or templates. Controls:

  • Isolate transport: export over an out-of-band management network or a dedicated VRF, never across the internet in clear text.
  • Restrict sources: collectors accept export only from known exporter addresses, enforced by host firewall and network ACLs.
  • Use secure transport where available: RFC 7011 covers TLS and DTLS with mutual authentication; IANA assigns port 4739 for IPFIX and 4740 for IPFIX over TLS or DTLS. Support varies widely by device, so check before designing around it.
  • Protect the collector: harden it like any log platform, with role-based access, because flow history is valuable reconnaissance for an intruder.
  • Watch for gaps: an exporter that goes silent can be a failure or a deliberate attempt to blind monitoring.

Where identity fits

Flows answer "which IP", and investigations need "which identity". Map IPs to users through VPN and DHCP logs, and to workloads through cloud instance and pod metadata (the AWS instance-id and pkt-srcaddr fields help). Service accounts and workload identities generate most east-west traffic, so baselining flows per workload makes misuse of a stolen machine credential visible: a build agent suddenly talking to a database it never used is a strong signal. Collector and exporter management accounts, and any API keys used to pull cloud flow logs, are themselves privileged non-human identities. See the Cloud Logging and Audit Trail Guide.

Detections and measurement

BehaviourFlow patternATT&CK
Internal scanningOne source, many destinations or ports, SYN-only or rejected flowsT1046 Network Service Discovery
BeaconingRegular, small flows from one host to the same external destination; see the Beaconing and C2 Detection GuideT1071 Application Layer Protocol; T1571 Non-Standard Port
Bulk exfiltrationUnusual outbound byte volume or out/in ratio to a new destinationT1041 Exfiltration Over C2 Channel; T1048 Exfiltration Over Alternative Protocol
Tunnelling over unusual protocolsLarge or persistent ICMP or other non-TCP/UDP flowsT1095 Non-Application Layer Protocol
Proxy and relay useInternal host relaying traffic between other hosts and the internetT1090 Proxy
Denial of serviceSudden packet or flow rate spikes to one destinationT1498 Network Denial of Service

Metrics:

  1. Percentage of zone boundaries and cloud VPCs exporting flow (target 100 per cent of in-scope).
  2. Exporter health: number of exporters silent for more than the expected interval.
  3. Template decode errors per day at the collector.
  4. Percentage of flows enriched with an asset or identity owner.
  5. Median time from flow export to availability in the SIEM.

Using flow data in investigations

During an incident, flow history is often the only record of what a compromised host talked to before the endpoint agent was installed or after it was disabled. Typical questions it answers quickly:

  • Scope: which internal hosts did the compromised system connect to, on which ports, and when did that start?
  • First contact: when did any internal host first talk to a known malicious address or a newly seen destination?
  • Volume: roughly how much data left through a given destination, which feeds breach notification decisions. Flow volume shows bytes moved, not what they contained.
  • Containment check: after blocking, did traffic to the attacker's infrastructure actually stop, or move to a new address?

Keep timestamps in UTC across exporters and collectors, and record exporter clock sources, because flow timelines are joined with endpoint and identity logs. The Data Exfiltration Playbook shows where flow evidence fits in a response.

Common failure patterns

  • Edge only: exporting at the internet firewall but not internally, so lateral movement between segments is invisible.
  • Silent heavy sampling: sampling at high ratios without recording the rate, then treating counts as exact.
  • IPv4-only templates: IPv6 traffic not exported at all, leaving a blind protocol.
  • Unprotected export: UDP export across untrusted networks, or collectors open to any source.
  • No enrichment: analysts staring at IP pairs with no owner, so alerts are never closed.
  • Ignoring cloud: on-premises flow is collected but VPC flow logs are off or use the default format without NAT-translated addresses.

Framework mapping

FrameworkReferenceHow it applies
NIST SP 800-53 Rev 5SI-4 System Monitoring; AU-12 Audit Record Generation; SC-7 Boundary ProtectionGenerating and analysing flow records at boundaries
NIST CSF 2.0DE.CM-01; DE.AE-02Monitoring networks and analysing adverse events
CIS Controls v8.1Control 13 Network Monitoring and DefenseCollecting network traffic flow logs; see the CIS Control 13 Network Monitoring and Defense Guide
MITRE D3FENDD3-NTA Network Traffic Analysis; D3-NTCD Network Traffic Community Deviation; D3-PMAD Protocol Metadata Anomaly DetectionDefensive techniques built on flow data

Practitioner checklist

  • Inventory exporters and the flow formats and IPv6 support of each.
  • Export unsampled flow at the internet edge, zone boundaries and data centre cores where possible.
  • Enable cloud flow logs in every VPC with a custom format including original addresses and TCP flags.
  • Place collectors on a management network and restrict export sources.
  • Use TLS or DTLS for IPFIX where devices support it.
  • Enrich flows with asset, identity, DHCP and DNS data.
  • Deploy scanning, beaconing, exfiltration and tunnelling detections in alert-only mode, then tune.
  • Alert on silent exporters and template decode errors.
  • Set retention to cover your investigation window and test retrieval.

Standards and references

Related NHI Mgmt Group resources: Network Monitoring and Packet Capture Guide · NDR Guide · Beaconing and C2 Detection Guide · Data Exfiltration Guide

PDF for subscribers

Take this guide with you

Every section, table and checklist in one PDF, with links to the sources and related guides. Subscribe to our weekly NHI & AI Identity Journal and we’ll email you a download link.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Researched and written by the NHI Mgmt Group editorial team using AI tools, under the editorial direction of Lalit Choda. Facts are drawn from the sources listed on this page and checked by our automated quality process. Last updated 27 October 2026. Spotted an error? Use our contact page.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM and Agentic AI security. nhimg.org

PDF Downloads for Newsletter Subscribers

Get the guide

Enter your details and we’ll email you a download link.

The full guide is free to read online. The PDF is a thank you for subscribing.

Fill in all fields marked * and tick the PDF box to continue.