Proving the Path Part 6: Egress, NPU Offload, and the Verdict

18 September 2026 fortigatefortiossdwantroubleshootingproving-the-pathnpu-offloaddiagnosereference

Part 5 confirmed the pinned session’s policy, NAT and IPsec tunnel were all correct and unbroken, on a tunnel that was briefly carrying more loss than a voice codec tolerates. The last stage, egress and NPU offload, is where the direct evidence for that loss actually lives, and it’s also where the series closes out with a verdict and a fix.

Getting visibility into an offloaded flow

Part 2 flagged that this session’s npu_state=0x4000 means it’s NPU-offloaded, which is exactly why a plain diagnose sniffer packet on the WAN interface during the incident wouldn’t have shown much: the ASIC is forwarding it without punting packets back to the CPU. Proving what actually happened to this flow means going to the counters that see traffic after it’s handed to hardware, not the ones that only see what reaches the CPU:

diagnose npu np7 anomaly-drop-counter
diagnose sys sdwan health-check

The health-check figures from Part 4 are, in fact, the right evidence here: they’re generated from probes riding the same physical path and NPU pipeline as the offloaded call, so a loss spike on dce-probe for to-hub1 during the incident window is a reliable proxy for what the Teams flow itself experienced on that same egress, at that same time. A CPU-side capture would have told me nothing useful; the health-check counters told me everything.

Reassembling the whole picture

Six stages, and each one contributed exactly one fact:

  • Ingress (Part 2): clean, ruled out immediately.
  • Session (Part 2): the call’s session predates the SLA breach by several minutes; it made its decisions once and hasn’t revisited them.
  • Policy route and routing (Part 3): correct right now, for a new flow, which is a different claim from what the old flow is doing.
  • SD-WAN and SLA (Part 4): worked exactly as designed, failing new flows over and back, with zero mechanism to touch a session already in progress.
  • Policy/NAT and IPsec (Part 5): unchanged and cryptographically healthy, on a tunnel that was briefly failing an SLA it doesn’t itself measure.
  • NPU offload (this part): hid the flow from CPU-level tools, which is why nobody’s quick sniffer check during the incident found anything.

None of those six facts individually explains a call staying choppy after the dashboard recovered. Together, they do: the call was pinned to HUB-1 before the breach, SD-WAN correctly protected every subsequent new call but had no way to touch this one, and the degradation itself was invisible to anyone checking tunnel state, member liveness, or a CPU-side sniffer, because none of those are the layer where a brief SLA breach actually shows up.

The fix

Once the hypothesis is this specific, the fix is narrow. Not clearing the whole session table and hoping: that destroys evidence for the next occurrence and resets far more than the one flow that needs it. The right move is a precisely filtered clear on the exact session:

diagnose sys session filter src 10.30.10.55 dport 3478
diagnose sys session clear

That forces just this flow to re-evaluate policy route, routing, SD-WAN and NAT from scratch, which, given Part 3’s finding that a fresh flow resolves correctly right now, sends it over the healthy path without touching anything else on the box. For a live call, clearing the session still glitches the audio for a moment, so in practice the better answer is often simpler: know that the call will recover on its own once it ends and a fresh one is placed, and use the targeted clear only when a session is pinned to a genuinely still-bad path, not a recovered one.

The full list of false conclusions this series is built on

Every part flagged one of these against the fault. Collected together, they’re the checklist I actually use:

  • The tunnel is up, so VPN is healthy. A phase 1/2 SA can be perfectly intact while the path behind it fails an application’s latency, jitter or loss tolerance.
  • The member is alive, so SD-WAN should use it. Link-alive and SLA-eligible are different tests; the second is what a service rule actually acts on.
  • The route exists, so traffic must use it. A route or policy route only governs sessions evaluated after it’s in place, never a session that’s already pinned.
  • The sniffer shows no traffic, so none exists. NPU-offloaded flows can be invisible to CPU-level capture tools long after they were handed to hardware.
  • A config line proves the policy matched. Configuration shows what would match now; the session table shows what actually matched, and when.
  • Clearing everything fixed it. It also destroys the evidence that would have told you why, and can mask a recurrence you’ll see again next week without understanding it.
  • Ping works, so the application path is fine. ICMP’s size, priority and state handling differ enough from a real application’s traffic that a healthy ping proves very little about it.

Where this leaves the series

Nine stages, six false conclusions, one fault. The methodology underneath all of it is the same sentence repeated at every stage: match the evidence to the specific question that stage answers, and don’t let a plausible story at one stage stand in for proof at another. Session pinning was the thread through this particular fault, but the discipline generalises. The next fault I chase down this chain will pin the blame somewhere else entirely, and the only way to find out where is to keep asking the same narrow questions in order.

If you’re working through a live fault of your own, Part 1 has the full nine-stage chain, and A Day in the Life of a Packet on a 50G FortiGate has the architecture and the command cookbook behind each stage in far more depth than this series went into.