tcpdump Deep Dive: BPF Filters, Capture Rotation, and Cross-Mapping to FortiGate's diagnose sniffer packet

26 April 2026 linuxnetworkingtcpdumpfortigatepacket-capturebpftroubleshooting

Why tcpdump, still

Wireshark is brilliant on a workstation. It is no help when you are five SSH hops deep on a customer jump box that has no GUI, no internet egress, and a pcap quota measured in megabytes before someone notices. tcpdump is what you reach for when the box is small, the link is hostile, and you need an answer in the next ten seconds.

This post is aimed at engineers who already know tcpdump -i eth0 host x.x.x.x and want to move past it. We will work through the parts of the command that matter under pressure, the BPF filter primitives that separate guesses from answers, the rotation patterns you need for anything that runs longer than a coffee break, and a clean cross-reference to FortiGate’s diagnose sniffer packet so you do not have to think twice when you swap from a Linux box to a firewall.

The anatomy of a tcpdump command

A useful invocation has four parts: where to capture, what to capture, how to display it, and where to put the bytes.

tcpdump -i eth0 -nn -s 0 -w /var/tmp/cap.pcap 'tcp port 443 and host 10.0.0.5'

The flags worth keeping in muscle memory:

-i <iface> — interface. -i any works on Linux and gives you a synthetic link layer. -D lists what is available, including loopback and anything you might have forgotten about (bond0, vlan100, wg0).
-nn — never resolve names or port numbers. Resolution is slow, and on a broken network it makes tcpdump hang while it tries DNS. Always use it.
-s 0 — snaplen. On modern tcpdump this defaults to 262144, but on older builds it still defaults to 96, which truncates payloads. If you are reading other people’s pcaps and find them mysteriously useless, the snaplen was probably wrong.
-w <file> — write raw frames to a pcap file. No display, lower CPU, no parsing surprises later.
-r <file> — read it back. Filters reapply at read time, so tcpdump -r cap.pcap 'tcp port 443' is fine.
-vvv — verbosity. Three v’s is the loudest, useful when you actually want to see decoded protocol content rather than archive bytes.
-e — show link-layer headers. The first thing you reach for when MAC addresses, VLAN tags, or frame types are part of the problem.
-X and -A — hex+ASCII or ASCII-only payload. Useful for clear-text protocols and for spotting human-readable artifacts in supposedly opaque traffic.
-c <n> — stop after n packets. The single most useful flag for capturing without filling a disk.

A small but important note on permissions: you do not need root to run tcpdump. The clean way is to grant the binary the capabilities it actually needs:

sudo setcap cap_net_raw,cap_net_admin+eip $(which tcpdump)

Now any user in the right group can capture. This matters when you are baking captures into automation that you would rather not run as root.

BPF filter syntax beyond host and port

The filter is the part that separates a useful capture from a four-gigabyte mystery. Berkeley Packet Filter expressions compose primitives with and, or, and not, and you can be much more precise than most people use them for.

The basic primitives:

host 10.0.0.5, src host 10.0.0.5, dst host 10.0.0.5
net 10.0.0.0/24
port 443, portrange 1000-2000, dst port 53
tcp, udp, icmp, arp, ip, ip6

Combine them properly with explicit grouping. This is fine:

host 10.0.0.5 and (port 443 or port 80)

This is a trap:

host 10.0.0.5 and port 443 or port 80

The second one is “host 10.0.0.5 and port 443” OR “any port 80 traffic”. You will capture the entire web. Always parenthesise.

Where BPF gets interesting is byte-offset filters. They let you reach into header fields directly:

# Capture only TCP SYNs (no SYN-ACKs)
tcpdump -i any 'tcp[tcpflags] & tcp-syn != 0 and tcp[tcpflags] & tcp-ack == 0'

# Capture TCP RSTs and FINs only — useful when something is killing connections
tcpdump -i any 'tcp[tcpflags] & (tcp-rst|tcp-fin) != 0'

# Capture ICMP echo requests only
tcpdump -i any 'icmp[icmptype] = icmp-echo'

# Capture HTTP GET requests on the wire (matches "GET ")
tcpdump -i any -A 'tcp port 80 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420'

The HTTP one looks frightening but is just “starting at the TCP payload, the first four bytes are G, E, T, space”. The (tcp[12:1] & 0xf0) >> 2 part calculates the TCP header length so we can skip past it.

VLANs are a perennial gotcha. By default a filter like tcp port 443 will not match traffic carrying an 802.1Q tag because the IP header is four bytes deeper. You either prefix the filter with vlan to shift the offset, or you use vlan and tcp port 443. If you are debugging a trunk and getting nothing, this is usually why.

Production-grade capture patterns

Anything you intend to leave running needs rotation. Otherwise you will discover at 03:00 that the disk is full and the captures you actually wanted have been overwritten. tcpdump has built-in rotation; use it.

tcpdump -i eth0 -nn -s 0 \
        -W 24 -G 3600 \
        -w '/var/captures/eth0-%Y%m%d-%H%M%S.pcap' \
        'not port 22'

Read this as: capture to a new file every 3600 seconds (-G), keep at most 24 of them (-W), embed timestamps in the filename, and exclude SSH so we do not record our own session. That is a 24-hour rolling window with one file per hour.

Size-based rotation is the alternative when traffic volume is variable:

tcpdump -i eth0 -nn -s 0 -C 100 -W 50 -w /var/captures/eth0.pcap 'not port 22'

-C 100 rotates every 100 MB. -W 50 keeps 50 files. With a busy link this can give you better resolution than time-based rotation.

A few more knobs that matter under load:

-B 4096 — capture buffer in KiB. If you are seeing dropped packets in the summary tcpdump prints when it exits, raise this. The default is small.
Drops are reported as N packets dropped by kernel. Watch for them. A capture with drops is unsafe to draw conclusions from for anything sequence-sensitive.
Always pin your filter as tightly as you can on the capture side. Display-side filtering is convenient but you have already paid for the disk and the buffer pressure.

Reading captures without Wireshark

You do not always have a workstation handy, and even when you do, you can answer most questions on the box itself.

# Reapply filters at read time
tcpdump -nn -r cap.pcap 'host 10.0.0.5 and tcp port 443'

# tshark gives you Wireshark display filters from the CLI
tshark -r cap.pcap -Y 'http.request' -T fields -e ip.src -e http.host -e http.request.uri

# Top talkers by packet count
tcpdump -nn -r cap.pcap 'ip' | awk '{print $3}' | cut -d. -f1-4 | sort | uniq -c | sort -rn | head

# Count TCP retransmits roughly
tshark -r cap.pcap -Y 'tcp.analysis.retransmission' | wc -l

tcpflow is worth knowing about for clear-text protocols — it reassembles TCP streams into one file per direction per flow, which is often what you actually want.

The FortiGate equivalent: diagnose sniffer packet

FortiOS does not give you tcpdump. It gives you diagnose sniffer packet, which is essentially the same idea wrapped in a more constrained CLI. Once you see the mapping it is the same tool with a different shell.

diagnose sniffer packet <interface> '<filter>' <verbosity> <count> <timestamp>

<interface> — any works, or a specific name like port1, wan1, internal.
<filter> — quoted BPF expression, mostly the same syntax as tcpdump.
<verbosity> — 1 through 6. The numbers matter: 4 gives you headers and payload (most common), 6 adds the interface name. Use 4 unless you have a reason.
<count> — packet count, 0 means run until you stop it.
<timestamp> — a for absolute UTC, l for local time. Always pass one or you get relative timestamps that are useless when correlating to logs.

Some examples that map directly onto things you would do in tcpdump:

# tcpdump: tcpdump -i any -nn 'host 10.0.0.5 and port 443'
diagnose sniffer packet any 'host 10.0.0.5 and port 443' 4 0 a

# tcpdump: tcpdump -i eth0 'icmp'
diagnose sniffer packet port1 'icmp' 4 0 a

# tcpdump: tcpdump -i any 'tcp[tcpflags] & tcp-syn != 0'
diagnose sniffer packet any 'tcp[tcpflags] & tcp-syn != 0' 4 0 a

# Stop after 100 packets and timestamp absolutely
diagnose sniffer packet any 'host 1.2.3.4' 4 100 a

To turn the output into a pcap file you can open in Wireshark, save the CLI session and pipe it through fgt2eth.pl (Fortinet have published the script and there are reliable community ports). Once you have it on your path, the workflow is: run the sniffer, capture the terminal output to a file, run fgt2eth.pl -in fgt.txt -out fgt.pcap, open in Wireshark.

A side-by-side cheat sheet, because no one remembers which order the FortiGate args come in:

What you want	Linux tcpdump	FortiOS diagnose sniffer packet
All traffic to/from a host	`tcpdump -i any -nn host 10.0.0.5`	`diagnose sniffer packet any 'host 10.0.0.5' 4 0 a`
Specific port both directions	`tcpdump -i any -nn 'port 443'`	`diagnose sniffer packet any 'port 443' 4 0 a`
ICMP only	`tcpdump -i any -nn 'icmp'`	`diagnose sniffer packet any 'icmp' 4 0 a`
Capture N packets and stop	`tcpdump -i any -c 100 ...`	`... 4 100 a`
Hex+ASCII payload	`tcpdump -X ...`	verbosity `6`
Save to pcap	`-w cap.pcap`	log terminal, run through `fgt2eth.pl`
Exclude SSH from your own session	`'not port 22'`	`'not port 22'`

The two big differences in practice: FortiOS does not give you ring buffers — if you need long-running rotated capture, run a Linux box on a SPAN port instead — and the kernel visibility is different. Traffic that has been NATed or VPN-decapsulated will be visible at different points in the pipeline, which means picking the right interface (any versus a specific port versus the IPsec tunnel interface) matters more on FortiGate than it does on Linux.

Patterns that earn their keep

A few troubleshooting recipes that I keep reaching for, both on Linux and on FortiGate.

Asymmetric routing. Run the same filter on every interface in play and compare. If you see SYNs leaving on one interface and SYN-ACKs arriving on a different one, you have your answer. On FortiGate, run two diagnose sniffer packet sessions on the inbound and outbound interfaces and look for traffic on only one side.

MTU and PMTUD blackholes. Filter for ICMP type 3 code 4 (icmp[icmptype] = 3 and icmp[icmpcode] = 4). If you do not see them and applications are stalling on large transfers, something between you and the destination is dropping the fragmentation-needed messages. Common on misconfigured firewalls including, ironically, FortiGate policies that drop ICMP wholesale.

TLS handshake failures without decrypting. Capture port 443 and look at the ClientHello. SNI is in cleartext, the cipher list is in cleartext, the server’s certificate chain is in cleartext. You can diagnose ninety percent of TLS issues without ever touching a private key. tshark -r cap.pcap -Y 'tls.handshake.type == 1' -T fields -e tls.handshake.extensions_server_name will pull SNI values out of a pcap.

Slow consumer detection. A flood of TCP zero windows from the receiver means the application is not draining its socket buffer. tshark -r cap.pcap -Y 'tcp.window_size == 0' and look at the source. This shows up as inexplicable application slowness that is “not a network problem” — it is, in fact, the application’s problem, and you can prove it.

DNS. tcpdump -i any -nn 'udp port 53' is the first thing to run when a thing that should work does not. Half the time you find a hostname resolving to something nobody expected, or not resolving at all.

Where to stop

tcpdump is a tool for asking specific questions. If you find yourself running it for hours, drawing graphs, or trying to reconstruct application behaviour from raw frames, you have outgrown it. That is the point at which you reach for Zeek for protocol-aware logging, Suricata for signature matching, or a packet broker feeding a real capture appliance. Keep tcpdump for the pointed questions where you already have a hypothesis and need to confirm or reject it.

The next post in this series covers NETEM, which is the other half of the same coin: once you can see traffic precisely, you can also shape it precisely, and that is what makes a Linux box a serious lab tool.