Wireshark for Security Analysis Guide: Packet Capture, Filters and Investigations

Packet captures settle arguments that logs cannot, but a capture taken from the wrong point, with the wrong filter or without a legal basis creates more problems than it solves. Wireshark is the most widely used open source protocol analyser, and security teams use it to confirm alerts, reconstruct sessions and prove what actually crossed the wire. This guide covers the defender workflow: where to capture, how capture filters differ from display filters, following streams, decrypting TLS in a lab you own, automating with TShark, and handling captures as the sensitive data they are. It is written for SOC analysts, incident responders and network engineers, and it deliberately excludes offensive interception techniques.

Key takeaways

• Capture and analysis are separate jobs. dumpcap or TShark collects packets on a sensor; the Wireshark GUI is where you analyse them, ideally on a different, hardened workstation.
• Capture filters discard, display filters hide. Capture filters use libpcap (BPF) syntax and drop packets for good; display filters use Wireshark's field syntax and can be changed at any time.
• Capture point decides what you can prove. A SPAN port is flexible but can drop packets and strip VLAN tags; a tap is preferred when timing, completeness or evidential value matter.
• TLS decryption needs secrets you legitimately hold. A key log file from your own client works for modern (EC)DHE and TLS 1.3 sessions; an RSA server key does not.
• Treat every pcap as personal data. Captures hold credentials, tokens, IP addresses and content, so they need a purpose, access control, encryption at rest and a retention limit.

What the tool does

Wireshark dissects hundreds of protocols, either live or from saved files, in a three pane packet list, detail and bytes view. It reads and writes pcap and pcapng, decompresses gzip captures on the fly, supports decryption for protocols including TLS, IPsec, Kerberos, SNMPv3 and WPA/WPA2 when you supply keys, and exports to CSV, XML or plain text. It runs on Windows, Linux, macOS and the BSDs. The project began in 1998 as Ethereal and was renamed Wireshark in 2006.

The Wireshark FAQ is explicit that it "was not designed to be an intrusion detection system". It does not alert, it does not baseline, and it does not scale to continuous monitoring of a busy link. Its value is depth on a bounded set of packets once something else (an IDS alert, an NDR detection, a Zeek log or a user report) has told you where to look. See the NDR Guide and the Zeek Network Monitoring Guide for the continuous layer.

The components

Defender use cases

• Alert validation: an IDS or NDR alert names a flow; open the matching packets and confirm whether the payload, handshake or response supports the alert before escalating. See the IDS and IPS Guide.
• Incident reconstruction: follow the session that delivered a file or carried a command, export transferred objects for sandboxing, and build a timeline from frame timestamps. See the Network Forensics Guide.
• Cleartext credential and secret exposure: check whether service accounts, API keys or bearer tokens travel over HTTP, LDAP simple bind, FTP or Telnet inside your network. Finding them is a defender win: rotate the secret and fix the transport. See the API Key Management Guide.
• Encryption and certificate verification: confirm negotiated TLS versions, cipher suites and the certificates actually presented, which catches legacy TLS 1.0 clients and unexpected interception devices.
• Beaconing and exfiltration review: the Conversations window lists packets, bytes, relative start and duration per pair of endpoints, which helps confirm regular small connections or one large outbound transfer suggested by flow data.
• Teaching and detection engineering: captures from a lab let analysts learn protocol behaviour and test IDS or Zeek logic against known traffic. See the OSI and TCP/IP Models Guide and the Networking Fundamentals for Defenders Guide.

Capture filters vs display filters

The two syntaxes look similar and are the most common source of beginner mistakes. Capture filters are applied by libpcap before packets are stored, so anything excluded is gone. Display filters are applied to packets already captured and can be refined endlessly.

Note that the meaning of != changed in Wireshark 3.6: it now means "all not equal", so ip.addr != 10.0.0.5 excludes every packet involving that address. Check older internal playbooks written before 3.6 for filters that relied on the previous behaviour.

Following streams and exporting evidence

Follow Stream works for TCP, UDP, DCCP, TLS, HTTP, HTTP/2, QUIC, WebSocket, SIP and USB CDC. It applies a display filter for the selected conversation and shows the reassembled data as ASCII, hex dump, raw, YAML and other encodings, by direction or for the whole conversation. "Filter out this stream" does the opposite, which is useful when stepping through many sessions. For HTTP/2 and QUIC a substream selector chooses the stream ID inside the connection. Save the filtered packets to a new pcapng and record the filter used, so another analyst can reproduce the finding.

TLS decryption in a lab you own

Most traffic worth examining is encrypted, and Wireshark can only decrypt it with secrets you supply. The Wireshark TLS wiki lists three methods:

• Key log file (recommended): browsers and tools built on NSS, OpenSSL or BoringSSL (Firefox, Chrome, curl) write per session secrets when the SSLKEYLOGFILE environment variable is set. Point the preference tls.keylog_file ("(Pre)-Master-Secret log filename") at the file. This works with Diffie-Hellman key exchange, which covers modern sessions.
• RSA private key: only for SSLv3 to TLS 1.2, non (EC)DHE cipher suites, non resumed sessions and the server certificate's key. It does not work for TLS 1.3 and is increasingly irrelevant.
• Pre-shared key: for PSK cipher suites, entered in hex.

Use this only on clients and test services you own, in a lab or a sanctioned troubleshooting window. A key log file decrypts every session it records, so treat it like a credential: generate it for the test, store it with the capture under the same controls, and delete both when finished. Never set SSLKEYLOGFILE globally on production or user machines. If decryption fails, enable tls.debug_logfile and check that TCP reassembly is enabled. Enterprise-wide decryption is a different control with its own trade-offs; see the TLS Inspection Guide.

Automating with TShark

TShark uses the same dissectors and display filters, so a filter proven in the GUI can be scripted. Useful options from the TShark manual page: -r reads a file, -Y applies a display filter, -T fields with repeated -e prints chosen fields (for example -e ip.src -e tls.handshake.extensions_server_name), -z with -q produces statistics such as conversations or protocol hierarchy, and -b with -a gives ring buffers and stop conditions for live capture. A typical pattern is to run TShark over a folder of incident captures, emit CSV of TLS server names or DNS queries, and load the result into the SIEM or a notebook.

Deployment and safe use

1. Define the purpose and approval first. Write down why you are capturing, which segment, for how long, who approved it and how long the data is kept. Capturing traffic you are not authorised to see can breach employment, privacy and interception laws; take legal advice for monitoring of staff. See the Employee Monitoring and Privacy Guide.
2. Choose the capture point. Capturing on the endpoint is easiest but sees one host. On switched networks use port mirroring (SPAN) or a tap. The Wireshark wiki notes that the mirror port must be at least as fast as the monitored ports, that switches can drop packets under load or strip 802.1Q tags, and that taps are generally preferred for compliance, timing analysis and malformed frames. ARP poisoning and MAC flooding disrupt networks and have no place in a defensive capture plan.
3. Minimise privilege. Elevate only dumpcap, never the GUI. On Linux add analysts to the wireshark group or grant dumpcap cap_net_raw and cap_net_admin with setcap; in containers add only NET_RAW and NET_ADMIN. On Windows install Npcap restricted to Administrators. Running Wireshark as root or Administrator exposes hundreds of dissectors, parsing hostile input, to full privilege.
4. Capture narrowly and bounded. Use a capture filter for the host or port in scope, and a ring buffer (-b filesize: and -b files:) or autostop (-a duration:) so the capture cannot fill the disk or run forever.
5. Analyse away from the sensor. Move files to an isolated analysis workstation that is patched to the current stable release. Captures from an incident may contain deliberately malformed packets aimed at dissector bugs.
6. Protect and retire the data. Hash each file on collection, store it encrypted with access limited to named analysts, log access, and delete it at the end of its retention period. Under GDPR, data minimisation and storage limitation (Article 5) and security of processing (Article 32) apply, because IP addresses, user names and content are personal data. See the Data Retention and Secure Deletion Guide.
7. Roll back cleanly. When the capture ends, remove the SPAN session or tap, revoke temporary group membership, and delete any key log files.

Licensing

Wireshark is free software under the GNU General Public License version 2. Using it to analyse your own or your customers' networks is permitted without a licence fee. The FAQ notes that modified versions distributed with a product must remain under the GPL, and recommends keeping Wireshark separate and communicating through pipes or sockets if you build tooling around it.

Detection and hardening value

Wireshark produces evidence rather than detections, but what analysts find in captures should feed controls and detection content.

Framework fit: packet analysis supports NIST CSF 2.0 DE.AE-02 (adverse events are analysed) and RS.AN-03 (establish what happened and the root cause); NIST SP 800-53 SI-4 System Monitoring and AU-6 review; AU-9 and AU-11 for protecting and retaining the capture itself; ISO/IEC 27001:2022 Annex A 8.16 Monitoring activities; and CIS Controls v8.1 Control 13 Network Monitoring and Defense. In D3FEND it is a tool for D3-NTA Network Traffic Analysis.

Measuring the capability

• Median time from alert to packet level confirmation for escalated network alerts.
• Percentage of high severity network alerts where packets or a Zeek record were available.
• Packet drop rate reported by dumpcap or the switch for each sanctioned capture.
• Number of cleartext secrets found in captures, and median time to rotation.
• Percentage of stored captures with a recorded purpose, owner, hash and deletion date.

Alternatives

Small vs large organisations: a small team can run Wireshark ad hoc with tcpdump or dumpcap on demand and a written approval. A larger SOC should keep Wireshark as the analyst's microscope on top of Zeek logs and indexed full packet capture, so that packets for an alert can be pulled after the event rather than captured only if someone happened to be watching.

Practitioner checklist

• Write a capture procedure covering purpose, approval, scope, duration, retention and deletion.
• Agree sanctioned capture points (taps or SPAN sessions) for core segments before an incident.
• Elevate only dumpcap; restrict Npcap to Administrators on Windows.
• Always use a capture filter and a ring buffer or autostop on live captures.
• Keep Wireshark on analysis workstations at the current stable release, isolated from production.
• Hash, encrypt and access control every stored capture; delete it on schedule.
• Restrict TLS key logging to lab clients and delete key logs with the capture.
• Turn every cleartext secret found into a rotation ticket and a transport fix.
• Script repeat analysis with TShark field output and share the filters with the team.

Standards and references

• Wireshark: User's Guide, Building Display Filter Expressions
• Wireshark: User's Guide, Filtering While Capturing
• Wireshark Wiki: TLS
• Wireshark Wiki: Ethernet Capture Setup
• Wireshark Wiki: Capture Privileges
• Wireshark: TShark Manual Page
• Wireshark: Frequently Asked Questions
• NIST: SP 800-53 Rev. 5

Related NHI Mgmt Group resources: Network Monitoring and Packet Capture Guide · Network Forensics Guide · Threat Hunting Guide · Digital Forensics (DFIR) Guide