FortiGate BGP Cookbook Part 2: Verify and Debug

Part 1 was the config. This is the other half: the commands I run after making a change, and the order I run them in when a prefix is missing or a session is down.

The same caveat as Part 1 applies. I did not lab this pair of posts. The commands come from Fortinet’s CLI reference and knowledge base, and I describe what each should show instead of pasting output I did not capture. I use the Part 1 scenario throughout, so the neighbor addresses below are the same ones you saw there and the commands can be pasted as they are. If you want the general FortiOS diagnostics rather than BGP specifically, the CLI reference series covers those.

Read the session first

get router info bgp summary

Start here every time. For each neighbor you get the remote AS, how long the session has been in its current state, and the State/PfxRcd column. If that last column holds a number, the session is Established and the number is how many prefixes you accepted from that peer. If it holds a word (Idle, Connect, Active, OpenSent, OpenConfirm), the session is not up and that word is your first clue. A neighbor that shows “never” for up/down has never reached Established.

get router info bgp neighbors 203.0.113.1

The detail view: peer address, router ID, remote AS, state, message counters and the capabilities that were negotiated. When two peers disagree about a capability, this is where you see it.

Follow one prefix through the policy

Most “my route is missing” problems are a prefix that was removed by a route map. Four views let you find which side of which filter it fell on.

get router info bgp neighbors 203.0.113.1 received-routes
get router info bgp neighbors 203.0.113.1 route
get router info bgp network
get router info routing-table bgp
get router info bgp neighbors 203.0.113.1 advertised-routes

What each one shows:

Command	What it shows
received-routes	Everything the peer sent, before your inbound policy. Needs soft-reconfiguration enable on the neighbor, otherwise you get % Inbound soft reconfiguration not enabled.
route	The routes that survived your inbound policy. With no inbound filter it matches received-routes.
get router info bgp network	The BGP table, including routes BGP knows about but did not install.
get router info routing-table bgp	What actually reached the RIB. BGP routes are marked B.
advertised-routes	What you are sending that neighbor, after your outbound policy.

Reading them in order tells you where a prefix disappeared. In the Part 1 setup, ISP1 might send you a full table, and you would see all of it in received-routes but only 0.0.0.0/0 in route, because the DEFAULT-ONLY prefix list dropped everything else. That difference is the filter working. If a route is in route but missing from the routing table, the cause is further along: a better route from another source, or a next hop that cannot be resolved.

If something you expect to announce is missing from advertised-routes, check the sender before blaming the receiver. BGP only advertises what it can find in the routing table, so a network statement with no matching route announces nothing. That is why Part 1 uses a blackhole route for the /24. Fortinet’s guidance also mentions that setting network-import-check to disabled changes this behavior, so that BGP advertises network statements whether or not the prefix is in the routing table. I treat that as a deliberate design choice rather than a fix, because it means you can announce a prefix you have no way to deliver traffic for.

Apply a change without dropping the session

execute router clear bgp ip 203.0.113.1 soft in
execute router clear bgp ip 203.0.113.1 soft out
execute router clear bgp ip 203.0.113.1 soft
execute router clear bgp all soft out

in re-applies your inbound policy to what the peer sent, out re-sends what you advertise, and the bare soft does both. Drop the soft and you get a hard reset, which tears the session down and clears its routes.

Two details catch people out. The word “soft” in these commands does not mean soft-reconfiguration. For inbound changes, the FortiGate asks the peer to resend its routes (Route Refresh, RFC 2918) when the peer supports it, and falls back to its stored copy only if it does not. With soft-reconfiguration on, an inbound refresh reads that stored copy, so a change the remote side has made since will not show up until it resends. And outbound refreshes go out at the next advertisement interval, so a change can take a moment to appear on the other end.

Some changes need the hard reset. Timer changes only take effect after one, and a change to negotiated capabilities triggers one on its own.

The session will not establish

Match the state in get router info bgp summary to the likely cause, and work down the list.

State	What it usually means
Idle	The session has not started or is waiting to retry. Check that a route to the peer exists.
Connect	Waiting for the TCP handshake to complete. An authentication mismatch (TCP-AO or MD5) also looks like this, because the peer drops the SYN.
Active	TCP is not completing. Check reachability to the peer on TCP 179, anything in the path that filters it, and authentication.
OpenSent / OpenConfirm	TCP is up and the BGP OPEN exchange is in progress. A stall here points to mismatched parameters, such as the remote AS number.
Established	Up. PfxRcd is the prefix count.

For Connect and Active, the checks I run are:

1. Is there a route to the peer? A debug message of Sock Status: 113-No route to host means the FortiGate has no active route to the neighbor address. Add one, or fix the one that is down.
2. Is it directly connected? If the peer is not on a connected subnet, eBGP’s default TTL of 1 blocks it, and debug shows denied due to non-connected next-hop. That is the case for loopback peering, and the fix is set ebgp-enforce-multihop enable on the neighbor, as in the loopback peering post.
3. Does TCP 179 get through? Capture it:

diagnose sniffer packet any 'host 203.0.113.1 and port 179' 6 0 l

If you see SYNs leaving and nothing coming back, something is dropping them. That could be a firewall in the path, a local-in policy on either end, or an authentication mismatch. A peer that cannot validate your TCP-AO or MD5 signature discards your packets without reply, so the capture looks exactly like a blocked port. Check authentication on both ends before you go hunting through firewall policy.

4. Is authentication matched? This is the check I would make second on any new session, because the symptom hides it.

For TCP-AO, which Part 1 uses:

diagnose sys tcp-auth-options

The output shows the algorithm in use and the send and receive key IDs and validity windows for each authenticated neighbor. Compare it with the peer. A mismatch in algorithm, key string, lifetimes or key IDs makes the peer drop your SYN silently, and so do differences in how the two ends treat TCP options such as MSS, timestamps, SACK and window scaling. FortiOS assigns the send and receive IDs itself, so with a Cisco or Juniper peer, confirm the ID mapping on the peer’s side. If the output says tcb is not available, Fortinet’s fix is to recreate the key chain, reapply it to the neighbor with set auth-options, and confirm the TCP control block is active.

For MD5, a configured password appears as TCP option 19 in the SYN. If you see the option on your SYN and no SYN-ACK comes back, the peer is not matching it. Fortinet’s guidance is also that a FortiGate sitting between two MD5-authenticated eBGP peers does not strip option 19, so a transit firewall will not break it. If the password was changed on one side only, expect the session to reset almost immediately.

If TCP completes and the session still will not reach Established, the problem is inside BGP, and the debug output below is where the reason appears.

Run BGP debug, then turn it off

diagnose debug reset
diagnose ip router bgp all enable
diagnose ip router bgp level info
diagnose debug console timestamp enable
diagnose debug enable

You will see the finite state machine transitions for each neighbor, such as a session cycling through Active and Connect, plus socket errors and the reasons a session is rejected. The timestamp line is worth the extra command when you are correlating against a peer’s logs or a packet capture.

On FortiOS 7.2 and later you can narrow the output so it is not buried in other neighbors’ chatter:

diagnose ip router bgp set-filter neighbor 203.0.113.1
diagnose ip router bgp show
diagnose ip router bgp set-filter reset

show prints the filter currently in force and set-filter reset clears it.

Always finish with:

diagnose debug disable
diagnose debug reset

Debug left running on a busy edge adds load and floods the console, and it is easy to walk away from a session with it still enabled. I make turning it off the last line of whatever I paste into a change ticket.

A cheat sheet

The whole list in one place, using the Part 1 neighbor.

get router info bgp summary
get router info bgp neighbors 203.0.113.1
get router info bgp neighbors 203.0.113.1 received-routes
get router info bgp neighbors 203.0.113.1 route
get router info bgp neighbors 203.0.113.1 advertised-routes
get router info bgp network
get router info routing-table bgp
execute router clear bgp ip 203.0.113.1 soft in
execute router clear bgp ip 203.0.113.1 soft out
execute router clear bgp ip 203.0.113.1
diagnose sys tcp-auth-options
diagnose sniffer packet any 'host 203.0.113.1 and port 179' 6 0 l
diagnose debug reset
diagnose ip router bgp all enable
diagnose ip router bgp level info
diagnose debug console timestamp enable
diagnose debug enable
diagnose debug disable
diagnose debug reset

For IPv6 peers, the same views exist under get router info6 bgp and get router info6 routing-table bgp.

What I would do next

This pair covers the edge where you face a provider. The next steps from here are tuning session failure detection and recovery, which is BFD and convergence tuning in the SD-WAN series, and the policy tools you can build on top of what is here, communities in particular, which the BGP Deep Dive covers in Part 7.