NSE5 Part 9: Diagnostics and Troubleshooting

NSE5 Part 9: Diagnostics and Troubleshooting

Part 9 of the NSE5 series. FortiManager’s diagnostic surface is bigger than people expect — the GUI exposes maybe 20% of what’s available. This post is organised by the symptom you’d actually hit and the command(s) that resolve it. The exam tests roughly half of these directly.

The first three commands on any FortiManager

get system status
get system performance
diagnose system print sysinfo

get system status is the human-readable banner — version, serial, hostname, HA mode, ADOM count. get system performance shows CPU, memory, disk usage. diagnose system print sysinfo is the verbose form covering load average, swap, and process counts.

A FortiManager with consistently high CPU is almost always running a stuck install or a long-running script. Check diagnose dvm task list next.

Device-manager (DVM) diagnostics

The DVM subsystem is the heart of FortiManager — it owns device registration, install operations, and the device database. Eight commands cover most of its surface:

diagnose dvm device list
diagnose dvm adom list
diagnose dvm group list
diagnose dvm proc list
diagnose dvm task list
diagnose dvm task del <id>
diagnose dvm cdb check
diagnose dvm cdb upgrade
  • device list — every registered device, ADOM, and connection state.
  • adom list — every ADOM with its OS version family and device count.
  • proc list — every running DVM process and its target device.
  • task list — every recent install task with outcome.
  • task del — cancel a stuck task. Last resort; usually a stuck task means another admin is mid-install.
  • cdb check — integrity check across the configuration database.
  • cdb upgrade — force a database schema upgrade. After firmware updates this normally happens automatically; if for some reason it doesn’t, this triggers it.

FGFM tunnel troubleshooting

If a managed FortiGate shows offline or installs hang, FGFM is where you start.

diagnose fgfm session list
diagnose fgfm channel list
diagnose debug application fgfmsd 255
diagnose debug enable

diagnose fgfm session list shows each tunnel — state (UP, DOWN, RECONNECT), local/remote IPs, last keepalive timestamp, current channel ID. The channel list subcommand shows individual logical channels (control, config retrieve, install, FortiGuard relay) inside a session.

For the deep dive:

diagnose debug application fgfmsd 255
diagnose debug enable
# reproduce
diagnose debug disable

The 255 flag enables every FGFM trace category. Output is verbose — pipe to a file or wear a buffer-bigger terminal. Common patterns:

  • dial timeout — TCP/541 not reachable. Check the FortiGate side.
  • ssl handshake fail — the FortiGate doesn’t trust the FortiManager certificate. Re-register or check set fmg-source-ip.
  • auth fail — wrong serial number or registration password.
  • channel reset by peer — the FortiGate side closed. Check diagnose fgfm sessions on the FortiGate.

On the FortiGate, the mirror commands are:

diagnose fgfm sessions
diagnose debug application fgfmd 255

Run both sides in parallel for tunnel-flap debugging.

Install-failure forensics

When an install fails, four places to check, in this order:

1. The install preview. Saved per task and viewable from Device Manager → right-click → Install Wizard → View Install Log. Most install errors are visible in the preview if you scroll the right page.

2. The install log.

diagnose dvm task list

Pick the task ID for the failed install; the log path is shown. The log is a stream of FortiOS CLI commands and their responses, including the line where the failure happened.

3. The deploymanager state.

diagnose test deploymanager getcheckstatus <device-name>
diagnose test deploymanager fixflag set <device-name>
diagnose test deploymanager fixflag unset <device-name>

getcheckstatus returns the deployment check state for a device. fixflag clears a stuck “device is being deployed to” lock, useful when an install died mid-flight and the device is now wedged.

4. On the FortiGate.

diagnose debug cli 7
diagnose debug enable
# trigger the install from FortiManager

Shows every CLI line FortiManager sends as the FortiGate executes it. The line that errors is the failure point.

OFTP and oftpd

OFTP (TCP/514) is the protocol for log forwarding to FortiAnalyzer and for fabric handshakes between FortiManager and other Fabric members. The FortiManager oftpd daemon listens on this port. If FortiAnalyzer features are enabled on FortiManager, log feeds from managed FortiGates land here.

diagnose system process pidof oftpd
diagnose debug application oftpd 255
diagnose debug enable

Common pattern: “FortiAnalyzer integration shows no logs”. Check oftpd is running (pidof oftpd returns a number), check TCP/514 is open in any firewall in front of FortiManager, check the FortiGates’ log forwarding config points to FortiManager’s IP.

FortiGuard troubleshooting

diagnose fmupdate fds-version
diagnose fmupdate fmg-stats
get fmupdate analyzer virusreport

fds-version is the single most useful command — every database with current version, build time, and download status. If a column says not received, FortiManager can’t reach Fortinet’s CDN. Check DNS, NTP, publicnetwork, and any upstream firewall.

execute fmupdate update-now

Force an immediate FortiGuard download. Useful for verifying the path works after a fix.

Packet capture

FortiManager has a CLI sniffer just like FortiOS:

diagnose sniffer packet port1 'tcp port 541' 4 0 a

Same syntax as FortiGate. Capture the FGFM handshake to confirm the TLS exchange completes; capture port 514 for OFTP issues; capture port 8890 for FortiGuard relay.

Memory and disk troubleshooting

A FortiManager whose disk fills up stops accepting installs and silently drops log entries. The two commands:

diagnose system disk monitor
diagnose system disk usage

disk monitor returns a percentage per partition. usage is the verbose breakdown by directory. The two big consumers are /drive0/fmgmt/ (the database) and /var/log/ (logs and traces).

Cleanup options:

  • Reduce revision retention (adom-revision-max).
  • Trim logs (config system log fortianalyzer setting).
  • Clear task history (diagnose dvm task del all — destructive, leaves no audit trail).

When all else fails — debug application haperiod

diagnose debug application haperiod -1
diagnose debug enable

The HA period debugger surfaces sync, election, and split-brain errors that don’t appear elsewhere. Use it when get system ha status says everything is fine but the secondary disagrees.

Common exam scenarios

  • “Device offline; FGFM session shows RECONNECT repeatedly.” Almost always TLS handshake failure — re-register with the correct serial.
  • “Install fails on one device, succeeds on others.” diagnose test deploymanager getcheckstatus will show a stuck flag; fixflag unset clears it.
  • “FortiGuard updates downloaded but not redistributed.” service-access fgtupdates not enabled on the FortiManager interface managed FortiGates use.
  • “FortiAnalyzer features active but no logs visible.” oftpd not running, or TCP/514 blocked.

Part 10 closes the series with the advanced features — FortiGuard distribution, scripting, fabric authorisation, and the FortiAnalyzer integration that makes a FortiManager more than a config pusher.