Nmap and the Scripting Engine: A Network Engineer's Field Guide to NSE

Most network engineers know Nmap exists and have run nmap -sV target at some point to answer “what’s listening on this box.” That’s a fraction of what it does. The scan engine underneath supports half a dozen fundamentally different probing techniques, and the Nmap Scripting Engine (NSE) turns it from a port scanner into a general-purpose network reconnaissance and validation framework — one that’s just as useful for confirming a firewall change did what you intended as it is for a security assessment.

This post covers Nmap properly: the scan types and what each one actually tests, timing and performance controls, the NSE category system, how to read and write NSE scripts, and a set of practical recipes for using Nmap as a network engineering tool rather than purely a security one.

A note on scope before any of this: only scan hosts and networks you own or have explicit authorization to test. Everything in this post assumes a lab or an estate you’re responsible for.

Scan Types: What Each One Actually Tests

Nmap’s scan types differ in which TCP/IP behaviour they probe, not just in speed. Picking the wrong one for the situation gives you confidently wrong answers.

TCP SYN scan (-sS) — the default, and usually the right choice

sudo nmap -sS 192.168.1.0/24

Sends a SYN, waits for SYN/ACK (port open) or RST (port closed), then sends a RST instead of completing the handshake. Fast, and because the handshake is never completed, it doesn’t get logged by naive application-layer logging the way a full connection would. Requires raw socket privileges (root / CAP_NET_RAW).

TCP Connect scan (-sT) — the unprivileged fallback

nmap -sT 192.168.1.0/24

Completes the full three-way handshake via the OS socket API. Slower, and it does show up in application logs and connection tables, but it works without elevated privileges — the only option on a box where you can’t get raw socket access.

UDP scan (-sU) — slow, but necessary

sudo nmap -sU -p 53,123,161,500,4500 target

UDP has no handshake, so Nmap infers state from whether it gets a response, an ICMP port-unreachable (closed), or nothing (open|filtered — ambiguous, since UDP services often don’t respond to empty probes). This is why UDP scans take much longer and report more ambiguous states than TCP scans — the protocol itself doesn’t give you a clean signal. Worth running deliberately against anything running NTP, SNMP, IPsec/IKE, or DNS rather than skipping UDP because it’s slow.

ACK, FIN, Xmas, Null scans — firewall behaviour probes, not port state

sudo nmap -sA target      # ACK scan
sudo nmap -sF target      # FIN scan
sudo nmap -sX target      # Xmas scan (FIN+PSH+URG set)
sudo nmap -sN target      # Null scan (no flags set)

These don’t reliably tell you whether a port is open — they tell you whether a stateless packet filter is present and how it’s behaving. A stateful firewall (FortiGate, most modern stacks) will RST or drop these consistently regardless of port state, while an old stateless ACL might respond differently for filtered vs. unfiltered ports. Useful specifically for fingerprinting filter behaviour, not for general port enumeration — use -sS or -sT for that.

Version detection (-sV) and OS detection (-O)

sudo nmap -sV -O target

-sV probes open ports with protocol-specific handshakes to identify the actual service and version, not just the port number — distinguishing an SSH server actually running on 8080 from a misconfigured assumption based on port alone. -O fingerprints TCP/IP stack behaviour (window sizes, TTL, option ordering, ISN generation) against a database of known OS signatures. Both add scan time and both are probabilistic — treat the output as a strong hint, not ground truth, particularly against a host behind a FortiGate doing TCP normalization, which can alter the fingerprint Nmap sees.

Host discovery vs. port scanning

# Ping sweep only — no port scan at all
sudo nmap -sn 192.168.1.0/24

# Skip host discovery entirely, assume every host is up
# (necessary when ICMP is filtered but TCP ports respond)
sudo nmap -Pn target

-Pn is the flag people forget and then conclude a host “doesn’t exist” when it’s actually just not responding to the ICMP echo and ARP probes Nmap uses by default for discovery — a FortiGate with ICMP disabled on an interface produces exactly this false negative.

Timing and Performance

Nmap ships six timing templates trading speed against accuracy and stealth:

sudo nmap -T0 target   # Paranoid — one probe every 5 minutes
sudo nmap -T1 target   # Sneaky
sudo nmap -T2 target   # Polite — reduces load, plays nice with the target
sudo nmap -T3 target   # Normal — the default
sudo nmap -T4 target   # Aggressive — assumes a reasonably fast, reliable network
sudo nmap -T5 target   # Insane — assumes excellent network conditions

For an internal lab or an estate you control, -T4 is almost always the right call — it assumes you’re not trying to avoid an IDS and that your network can handle the parallelism. -T0/-T1 exist for situations where you genuinely need to avoid tripping a detection threshold, which in a legitimate context usually means a fragile legacy device that falls over under concurrent connections rather than anything adversarial.

Individual timing knobs, if a template doesn’t fit:

sudo nmap --min-rate 500 --max-retries 2 --host-timeout 30s target

--min-rate forces a minimum packets-per-second floor — useful when scanning a large subnet and the adaptive timing model is being more conservative than the network actually requires.

Output Formats

sudo nmap -oN scan.txt target      # Normal — human readable
sudo nmap -oX scan.xml target      # XML — structured, parseable
sudo nmap -oG scan.gnmap target    # Grepable — legacy, line-oriented
sudo nmap -oA scan target          # All three at once, same base filename

-oX is the one worth defaulting to for anything you intend to process programmatically — it’s structured and Nmap ships an XSL stylesheet alongside it for browser viewing, but more importantly it converts cleanly to JSON for the kind of jq-based filtering covered in the jq post:

# xmlstarlet or python's xml.etree can do this; a quick one-liner
# using a small Python helper since nmap has no native JSON output:
python3 -c "
import xml.etree.ElementTree as ET, json, sys
root = ET.parse('scan.xml').getroot()
hosts = []
for h in root.findall('host'):
    addr = h.find('address').get('addr')
    ports = [
        {'port': p.get('portid'), 'state': p.find('state').get('state'),
         'service': (p.find('service').get('name') if p.find('service') is not None else None)}
        for p in h.findall('.//port')
    ]
    hosts.append({'addr': addr, 'ports': ports})
json.dump(hosts, sys.stdout)
" | jq '.[] | select(.ports[].state == "open")'

Once it’s JSON, every pattern from the jq post applies directly — select() for filtering open ports, group_by() for grouping by service, map() for reshaping into a summary table.

The Nmap Scripting Engine

NSE scripts are written in Lua and hook into the scan at defined points. Each script declares a rule (when it runs) and belongs to one or more categories (how it’s classified). This is the part of Nmap that turns it from a scanner into a framework.

Categories

CategoryWhat it means
safeWon’t crash the target or consume excessive resources
intrusiveMay affect target performance or stability — use deliberately
vulnChecks for specific known vulnerabilities
exploitAttempts to actually exploit a vulnerability — never run outside an authorized test
discoveryGathers information — service details, directory listings, etc.
authTests for authentication bypass or weak/default credentials
bruteBrute-forces credentials — slow, and noisy on anything with lockout policies
defaultRuns automatically with -sC
versionUsed internally to assist -sV
dosMay cause denial of service — extreme caution
externalSends data to a third-party service (e.g. an online database)
# Run the default-safe script set
sudo nmap -sC target

# Run a specific category against a target
sudo nmap --script vuln target

# Run a specific script by name
sudo nmap --script ssl-cert,ssh2-enum-algos target

# Combine categories
sudo nmap --script "safe and discovery" target

The boolean expression syntax (and, or, not) on --script lets you compose categories precisely — "default or safe" is a common starting point that’s broader than -sC alone without straying into intrusive territory.

Scripts that earn a permanent place in a network engineer’s toolkit

# TLS certificate details — expiry, issuer, SANs, weak ciphers
sudo nmap --script ssl-cert,ssl-enum-ciphers -p 443 target

# SSH algorithm enumeration — useful for confirming hardening took effect
sudo nmap --script ssh2-enum-algos -p 22 target

# SNMP — community string brute force and walk (only with authorization,
# and only against your own devices)
sudo nmap --script snmp-info,snmp-sysdescr -sU -p 161 target

# HTTP service fingerprinting — titles, headers, common paths
sudo nmap --script http-title,http-headers,http-methods -p 80,443 target

# SMB — OS discovery and share enumeration on Windows/Samba hosts
sudo nmap --script smb-os-discovery,smb-enum-shares -p 445 target

# Banner grabbing across arbitrary ports
sudo nmap --script banner -p 21,22,23,25 target

The ssl-enum-ciphers script in particular is worth running as a matter of routine against anything terminating TLS — it grades each negotiated cipher suite and flags weak ones, which is the same information an SSL Labs-style audit gives you, run locally and without sending anything to a third party.

Writing a Custom NSE Script

A minimal script that connects to a port, sends a probe, and reports something specific to your environment — for example, confirming a FortiGate management interface is responding with the expected banner and not something unexpected (a sign someone repurposed the interface, or a rogue device took the IP):

-- fortigate-banner-check.nse
local nmap = require "nmap"
local shortport = require "shortport"
local stdnse = require "stdnse"

description = [[
Connects to a FortiGate HTTPS admin interface and checks the
response for the expected management banner string.
]]

author = "Micheal Garner"
license = "Same as Nmap -- See https://nmap.org/book/man-legal.html"
categories = {"discovery", "safe"}

portrule = shortport.port_or_service(443, "https")

action = function(host, port)
  local socket = nmap.new_socket()
  socket:set_timeout(5000)

  local ok, err = socket:connect(host, port, "ssl")
  if not ok then
    return nil
  end

  socket:send("GET / HTTP/1.1\r\nHost: " .. host.ip .. "\r\nConnection: close\r\n\r\n")
  local status, response = socket:receive()
  socket:close()

  if not status then
    return nil
  end

  if string.find(response, "FortiGate") then
    return "FortiGate management interface confirmed"
  else
    return "WARNING: port 443 responded but banner does not match FortiGate"
  end
end

Run it directly by path without installing it into the script database:

sudo nmap --script ./fortigate-banner-check.nse -p 443 192.168.100.1

The structure is the same for every NSE script: a portrule or hostrule function that decides whether the script applies to a given port/host, and an action function that runs the actual probe and returns a result. shortport.port_or_service is the standard helper for “does this look like the right kind of port” — it matches both by port number and by Nmap’s own service-detection guess, so the script still fires if the management interface has been moved to a non-default port and -sV correctly identified it.

Installing a script permanently

sudo cp fortigate-banner-check.nse /usr/share/nmap/scripts/
sudo nmap --script-updatedb
sudo nmap --script fortigate-banner-check -p 443 192.168.100.0/24

--script-updatedb rebuilds Nmap’s script database index so the new script is discoverable by name rather than only by path.

Practical Recipes

Validating a firewall policy change

After a FortiGate policy change, confirm only the intended ports actually changed state — directly comparable to the config-tree diffing approach in fgt-config-diff, but validated from the network’s perspective rather than the config’s:

# Before the change
sudo nmap -sS -p- -oX before.xml 10.0.1.50

# After the change
sudo nmap -sS -p- -oX after.xml 10.0.1.50

# Nmap's own diff tool
ndiff before.xml after.xml

ndiff ships with Nmap and produces a structured diff of two scans — new open ports, newly closed ports, and state changes — the same “show me what actually changed” instinct behind fgt-config-diff, applied to observed network behaviour instead of configuration text.

A scheduled drift-detection scan

#!/usr/bin/env bash
set -euo pipefail

BASELINE="/var/lib/nmap-audit/baseline.xml"
CURRENT=$(mktemp)

sudo nmap -sS -p- -oX "$CURRENT" 10.0.1.0/24

if [[ ! -f "$BASELINE" ]]; then
    cp "$CURRENT" "$BASELINE"
    echo "Baseline created."
    exit 0
fi

if ! diff_output=$(ndiff "$BASELINE" "$CURRENT"); then
    echo "Drift detected:"
    echo "$diff_output"
    exit 2
fi

echo "No drift."
rm -f "$CURRENT"

Exit code 2 on detected drift follows the same CI-gate-friendly convention used in pmtud-sweeper and rst-forensics — wire this into a scheduled task or a monitoring system that treats a non-zero exit as an alert, rather than parsing free-text output.

Cross-checking a scan against what’s actually listening

A remote Nmap scan tells you what’s reachable from the scanner’s vantage point — which is affected by every firewall and NAT rule in the path. Cross-reference against ss run locally on the target, covered in the ss post, to separate “not listening” from “listening but filtered”:

# From the scanner
sudo nmap -p 1-65535 -sS target

# On the target itself
ss -tlnp

A port that ss shows as LISTEN but Nmap reports as filtered confirms a firewall is the reason it’s unreachable, not a missing service — a meaningfully different diagnosis with a different fix.

Verifying scan results at the packet level

When a scan result looks wrong — a port reported open that shouldn’t be, or vice versa — tshark settles the argument, using the same display-filter approach from the ethtool/tshark post:

sudo tshark -i eth0 -f "host 10.0.1.50 and tcp" -Y "tcp.flags.syn==1" &
sudo nmap -sS -p 443 10.0.1.50

Watching the actual SYN/SYN-ACK/RST exchange removes any ambiguity about what Nmap inferred versus what happened on the wire — useful when a result needs to be defended in a change-review meeting.

A Note on Safety

vuln, brute, dos, and exploit category scripts are not toys. Running --script vuln against a fragile legacy device can crash it. Running brute against anything with an account lockout policy will lock out the accounts you’re testing. exploit scripts do exactly what the name says. Treat anything outside safe and default the way you’d treat a debug command on production gear — know exactly what it does before running it, and never run it against something you don’t own or have explicit written authorization to test.

  • Linux ss — the local-host counterpart to a remote Nmap scan; cross-referencing the two separates “filtered” from “not listening”
  • jq for network engineers — once Nmap’s XML output is converted to JSON, every filtering and reshaping pattern from this post applies directly
  • fgt-config-diff — the configuration-side equivalent of the ndiff drift-detection recipe above; pairs well for full before/after change validation