Text Processing for Network Engineers Part 4: awk and Turning Command Output Into Reports

grep finds lines. sed changes text. awk is the one of the three that thinks in columns, and that makes it the right tool for almost anything that looks like a table, a counter, or a report, once the raw text is on screen.

Fields and records

awk splits every input line (a “record”) into “fields” on whitespace by default: $1 is the first field, $2 the second, $0 is the whole line. That single idea covers most of what you’ll ever ask it to do:

ip -s link show eth0 | awk '/RX:/{getline; print $1, $3}'

Run against ip -s link’s output, that grabs the RX line’s byte and error counts by field position. The getline there pulls the next line after the match, because ip -s link’s header and data sit on separate lines, a good example of awk’s core skill: it doesn’t just match a pattern, it acts on the structure around the match.

Change the field separator with -F for anything that isn’t whitespace-delimited:

awk -F':' '{print $1}' /etc/passwd
awk -F',' '{print $2}' interface-inventory.csv

BEGIN, END, and the pattern-action model

An awk program is a series of pattern { action } pairs. BEGIN runs once before any input is read, END runs once after the last line, and everything else runs once per matching line:

awk 'BEGIN{total=0} /error/{total++} END{print "Total errors:", total}' switch.log

That’s a working error counter in one line: initialize a counter, increment it on every line containing “error,” print the final count. No pattern at all means “every line”:

awk '{print NR, $0}' config.txt          # NR is the built-in line-number variable

Associative arrays: the feature that turns awk into a reporting tool

awk’s arrays are keyed by string, not just integer index, which makes them a one-line group-by. Counting log entries by source IP:

awk '{count[$1]++} END{for (ip in count) print ip, count[ip]}' access.log

count[$1]++ increments a counter keyed on whatever the first field is, for every line. The END block then walks the array and prints every distinct key with its total. That’s “top talkers,” “which source IP hit this rule most,” or “which device sent the most syslog lines today,” all with the same three-line pattern, just changing what $1 points at.

Sort the result by piping into sort, since awk’s for (ip in count) has no guaranteed order:

awk '{count[$1]++} END{for (ip in count) print count[ip], ip}' access.log | sort -rn | head -10

Bandwidth and error-rate math from raw counters

This is the job awk was built for: show interface output is text, but the numbers inside it are numbers, and you usually want arithmetic on them, not just extraction.

Ethernet1/1 is up
  30 second input rate 125000000 bits/sec, 15000 packets/sec
  30 second output rate 98000000 bits/sec, 12000 packets/sec
  input errors 4, output errors 0

Pulling the input rate in Mbps and flagging anything over a threshold:

awk '/input rate/{gsub(/,/,""); print $4/1000000, "Mbps"}' show-interface.txt

gsub(/,/,"") strips the thousands-separator comma awk’s field splitter would otherwise choke on, then $4 is the raw bits/sec figure, divided down to Mbps inline. Add a conditional and this becomes an alert rather than just a printout:

awk '/input rate/{gsub(/,/,""); if ($4/1000000 > 800) print "HIGH:", $4/1000000, "Mbps"}' show-interface.txt

An error-rate table across every interface in a full show interface dump, keeping a running total per interface name:

awk '/^[A-Za-z].*is (up|down|administratively)/{iface=$1} /input errors/{errs[iface]=$3} END{for (i in errs) print i, errs[i]}' show-interface-all.txt

The first pattern catches interface header lines and remembers the current interface name in a variable; the second pattern, whenever it sees an errors line, records that count against whatever interface name is currently held. This is the awk idiom for parsing multi-line records that don’t arrive as one clean line: carry state across lines in a plain variable, and let END produce the summary once the whole file has been walked.

CSV out, for anything downstream

Set OFS (output field separator) and you get a clean CSV from ragged text with almost no extra code:

awk 'BEGIN{OFS=","} /input rate/{gsub(/,/,""); print iface, $4/1000000}' show-interface.txt > rates.csv

That CSV is now something a spreadsheet, a Python script, or a monitoring import job can consume directly, generated from raw CLI output with a single awk one-liner instead of hand-copying numbers into a spreadsheet.

Where awk stops making sense

Once the logic needs real control flow, nested data structures, error handling, or has to call out to an API, awk has technically got the syntax for loops and functions, but you’re fighting the tool at that point. That’s the line where this series’ Python posts take over (see the Python for Network Engineers series). The rule of thumb: if the whole job fits on one terminal line, or close to it, awk is usually right. If you’re writing a .awk script file with more than about thirty lines in it, it’s worth asking whether Python would be less code, not more.

Part 5 covers the smaller tools that live in the same one-liners as grep, sed, and awk: cut, sort, uniq, tr, and the rest of the supporting cast that turns these three into full pipelines.