Management Plane Separation on FortiGate SD-WAN: VRF vs Management VDOM

The question I keep getting asked

I have built management separation with VRFs on FortiGate SD-WAN designs, as in the VRF1 transport and loopback design and MP-BGP and VRFs on FortiGate SD-WAN, where VRF 20 carries FortiManager, FortiAnalyzer, SNMP and syslog. Other engineers I respect do the same job with a management VDOM. Both camps think the other is overcomplicating things.

The earlier VDOM post ended on a one-liner: VRF for traffic separation within a team, VDOM for administrative separation between teams. That holds for tenants. For the FortiGate’s own management plane it is too blunt, because “management separation” is really three separate problems, and the two approaches solve different subsets of them.

This is a no-lab post. The configuration comes from Fortinet’s documentation and from the design in my earlier VRF posts, and I have not run it against a specific build for this write-up. Where I am relying on behaviour I have not seen, I say so, and the end of the post lists what a lab should confirm.

Three questions hiding in “management separation”

  1. Local-out routing. FortiManager (FGFM), FortiAnalyzer logging, syslog, SNMP, RADIUS and LDAP for admin login, NTP, DNS and FortiGuard are all traffic the FortiGate originates itself. Which routing table do they use, and what source address do they carry?
  2. Inbound admin access. Who can reach the GUI and SSH, from where, on which interfaces?
  3. Administrative delegation. Which humans can see and change what? Can the team that owns the data plane be kept out of the management plane configuration?

A management VRF answers question 1 well. A management VDOM answers questions 1 and 3. Question 2 is answered in both designs by the same mechanisms: allowaccess on each interface, trusted hosts on each administrator, and local-in policy. Neither a VRF nor a VDOM gates it by itself. I think that is the most important thing to take from this post, and it is the part I would test first in a lab.

Approach A: the management VRF

The shape

The design I have used before keeps everything in one VDOM and uses VRFs as routing instances on the same hardware:

  • VRF 1 carries the transport and internet-facing traffic, including FortiGuard, NTP and DNS.
  • VRF 20 carries management: a management loopback, the NMS subnet, FortiManager and FortiAnalyzer.
  • The overlay carries VRF 20 end to end, so a spoke’s management loopback is reachable from the hub without touching the data VRFs. The VRF1 post has the MP-BGP detail.

Interfaces and loopback

config system interface
    edit "lo-mgmt"
        set vdom "root"
        set vrf 20
        set ip 10.255.20.1 255.255.255.255
        set allowaccess ping ssh https
        set type loopback
    next
    edit "port5"
        set vdom "root"
        set vrf 20
        set ip 10.20.10.1 255.255.255.0
        set allowaccess ping
        set role lan
    next
end

I would not put the management interface in VRF 0. Fortinet’s own troubleshooting tip describes VRF 0 as special: on 7.4.0, local-in and local-out traffic on an interface in VRF 0 does its routing lookup across routes from all VRFs, so a reply can leave through a different interface than the request arrived on. SSH and HTTPS break while ping keeps working. The tip says 7.2.10 limits VRF 0 to its own table, and recommends a non-zero VRF with its own default route for a management-only interface. VRF 20 does exactly that.

Pinning local-out traffic

This is the part that catches people. If a destination is reachable in more than one VRF, FortiGate picks the route in the lowest-numbered VRF. Fortinet’s example has DNS reachable through VRF 11 and VRF 30, and the FortiGate chooses VRF 11. Applied to this design: anything reachable in both VRF 1 and VRF 20 will go out of VRF 1 unless you say otherwise, and VRF 1 is the lower number.

From 7.6.1 some local-out services have a vrf-select option. The documented example is DNS:

config system dns
    set primary 10.20.10.53
    set vrf-select 1
end

I have only seen vrf-select documented for DNS. For the other services I would use the older, widely supported approach of pinning the interface and the source address per service, and check with set ? on the build in front of you which services also offer vrf-select:

config system central-management
    set type fortimanager
    set fmg "10.20.10.10"
    set fmg-source-ip 10.255.20.1
    set interface-select-method specify
    set interface "lo-mgmt"
end

config log fortianalyzer setting
    set status enable
    set server "10.20.10.11"
    set interface-select-method specify
    set interface "lo-mgmt"
    set source-ip 10.255.20.1
end

config log syslogd setting
    set status enable
    set server "10.20.10.12"
    set interface-select-method specify
    set interface "lo-mgmt"
    set source-ip 10.255.20.1
end

config user radius
    edit "NPS-MGMT"
        set server "10.20.10.13"
        set secret "example-secret"
        set interface-select-method specify
        set interface "lo-mgmt"
        set source-ip 10.255.20.1
    next
end

For SNMP the source address is set per community host (set source-ip under config hosts) or per v3 user. One gotcha from Fortinet’s source-IP tip: enabling ha-direct makes the FortiGate ignore source-ip for some services, including remote logging, and the CLI prompts you to unset them.

Two things are worth stating plainly about this list. Every service is a separate setting, so a new service added later is in whatever VRF the lowest-number rule gives it until someone remembers to pin it. And the split is by destination, not by “management or not”: in my design FortiGuard, NTP and DNS leave through VRF 1 while FortiAnalyzer and SNMP leave through VRF 20. That is correct, and it is also more to keep straight than a single “everything the box originates goes here” switch.

Inbound access

Admin access is controlled per interface and per administrator, not per VRF:

config system interface
    edit "port1"
        set allowaccess ping
    next
end

config system admin
    edit "netops"
        set accprofile "super_admin"
        set trusthost1 10.20.10.0 255.255.255.0
        set trusthost2 10.255.30.0 255.255.255.0
    next
end

config firewall local-in-policy
    edit 1
        set intf "lo-mgmt"
        set srcaddr "NET-MGMT-ADMIN"
        set dstaddr "all"
        set service "SSH" "HTTPS"
        set schedule "always"
        set action accept
    next
    edit 2
        set intf "lo-mgmt"
        set srcaddr "all"
        set dstaddr "all"
        set service "SSH" "HTTPS"
        set schedule "always"
        set action deny
    next
end

The VRF does not stop an admin session arriving on a data interface that has https in its allowaccess. Strip allowaccess from data and transport interfaces down to ping at most, and make lo-mgmt the only place SSH and HTTPS are accepted. My RADIUS admin auth post covers the RBAC profiles that sit on top of this.

Where the VRF approach is weak

  • No administrative boundary. Anyone who can administer the data plane can administer the management VRF, change the source-ip settings, or edit the local-in policy. For one team that is fine. For two teams it is the whole problem.
  • Leaks are routing decisions. If someone leaks routes between VRFs the separation is gone, and I have written about how route leaking behaves on a real build.
  • Services that ignore your intent. In the VRF1 post I noted that initial FortiCloud onboarding traffic uses VRF 0 regardless of vrf-select. Any service that does not honour the pinning follows the lowest-VRF rule.

Approach B: the management VDOM

What it moves

In multi-VDOM mode, Fortinet’s documentation defines the management VDOM as the VDOM where all the management traffic for the FortiGate originates. By default that is root. The documented list is DNS lookups, logging to FortiAnalyzer or syslog, FortiGuard services, alert email, NTP and SNMP traps. I would expect FortiManager connectivity to follow the same rule, but that is an expectation, not something I have seen documented or tested.

The help page I found for this is for FortiOS 5.2, so treat the details as long-standing behaviour that needs confirming on 7.6. It also says you cannot change the management VDOM while any administrator is using RADIUS authentication, and that the target VDOM should already exist and be able to reach the internet. If you authenticate admins against RADIUS, as in my earlier design, that restriction is a planning item.

The practical difference from the VRF approach is that you stop pinning services one at a time. You pick a VDOM and the box’s own traffic originates there.

Building it

config system global
    set vdom-mode multi-vdom
end

Enabling multi-VDOM mode on a live device restructures the configuration, so plan it as a maintenance window. Then create the management VDOM and an inter-VDOM link between it and the data VDOM:

config vdom
    edit "MGMT"
    next
end

config global
    config system vdom-link
        edit "vl-mgmt"
            set type ethernet
        next
    end

    config system interface
        edit "vl-mgmt0"
            set vdom "MGMT"
            set ip 10.255.254.1 255.255.255.252
            set allowaccess ping
        next
        edit "vl-mgmt1"
            set vdom "root"
            set ip 10.255.254.2 255.255.255.252
            set allowaccess ping
        next
    end
end

A management loopback and a default route toward the data VDOM go inside MGMT:

config vdom
    edit "MGMT"
        config system interface
            edit "lo-mgmt"
                set vdom "MGMT"
                set ip 10.255.20.1 255.255.255.255
                set allowaccess ping ssh https
                set type loopback
            next
        end
        config router static
            edit 1
                set gateway 10.255.254.2
                set device "vl-mgmt0"
            next
        end
    next
end

The data VDOM needs a route back to the management loopback and policies on its side of the link. This is the step the earlier VDOM post warned about: traffic crossing an inter-VDOM link is a new session to the destination VDOM and gets full policy evaluation.

config vdom
    edit "root"
        config router static
            edit 20
                set dst 10.255.20.1 255.255.255.255
                set gateway 10.255.254.1
                set device "vl-mgmt1"
            next
        end
        config firewall policy
            edit 100
                set name "MGMT-to-overlay"
                set srcintf "vl-mgmt1"
                set dstintf "OVERLAY"
                set srcaddr "MGMT-LOOPBACK"
                set dstaddr "NET-MGMT-SERVERS"
                set action accept
                set schedule "always"
                set service "ALL"
                set logtraffic all
            next
            edit 101
                set name "MGMT-to-internet"
                set srcintf "vl-mgmt1"
                set dstintf "UNDERLAY"
                set srcaddr "MGMT-LOOPBACK"
                set dstaddr "all"
                set action accept
                set schedule "always"
                set service "DNS" "NTP" "HTTPS"
                set nat enable
                set logtraffic all
            next
        end
    next
end

Then move the management role:

config global
    config system global
        set management-vdom "MGMT"
    end
end

Everything the box originates now starts in MGMT, crosses vl-mgmt, and is policed in root before it reaches the overlay or the underlay. The per-service source-ip settings still matter if you want a stable loopback source, but you no longer need interface-select-method on every service to keep it out of the data routing table.

Delegation, the actual prize

config system accprofile
    edit "prof_dataplane"
        set netgrp read-write
        set fwgrp read-write
        set sysgrp none
    next
end

config global
    config system admin
        edit "dp-engineer"
            set accprofile "prof_dataplane"
            set vdom "root"
            set trusthost1 10.20.30.0 255.255.255.0
        next
    end
end

A VDOM-scoped administrator works inside their VDOMs and cannot reach global settings such as the management VDOM setting, central management, or the admin accounts. They can alter the data plane without being able to redirect where the box sends its logs or which FortiManager it trusts. That is the property a VRF cannot give you, and it is the reason to choose this approach.

One caveat I flagged earlier: inbound reachability is still allowaccess on each interface, in whichever VDOM the interface lives. I believe a data-plane interface with https enabled still accepts admin logins whatever the management VDOM is, so the stripping of allowaccess from data interfaces applies here too. That is general FortiOS behaviour as I understand it, not something I have confirmed for this design.

Where the VDOM approach hurts in SD-WAN

  • An interface belongs to exactly one VDOM. The SD-WAN underlay and overlay live in the data VDOM. The management VDOM has no path of its own, so its traffic transits the data VDOM through the inter-VDOM link. You either accept that dependency or give the management VDOM its own physical or VLAN path, which means more interfaces and more circuits at every site.
  • SD-WAN configuration is per VDOM. The management VDOM normally runs none, but the data VDOM’s SD-WAN rules now steer management traffic too, and the resilience design has to account for it.
  • Management depends on the data plane’s health. If a policy on vl-mgmt1 is wrong, the box loses FortiManager and logging, and it may lose the path you need to fix it. The VRF design has the same dependency only if you route management through the same tunnels, which you can choose not to do.
  • FortiManager gains an axis. VDOMs are visible as separate managed entities. That is useful for delegation, and it is another layer in templates and policy packages across a large estate.
  • Licensing and platform limits. Included VDOM counts vary by model and extra VDOMs are a licence. Check the datasheet for the platforms you actually deploy, especially small branch units.
  • Conversion is disruptive. Enabling multi-VDOM mode and moving the management role are maintenance-window changes. Adding a VRF is not.

The third option, which is secretly the second

FortiGate has a feature for exactly this, on models with a dedicated management port. dedicated-mgmt gives you out-of-band management with its own routing table:

config system dedicated-mgmt
    set status enable
    set interface "mgmt1"
    set default-gateway 10.24.3.200
end

According to Fortinet’s technical tip, enabling it creates a hidden VDOM called dmgmt-vdom, moves the interface into it, and installs the default route inside that VDOM. You reach it with execute enter dmgmt-vdom. The interface can no longer be referenced elsewhere in the configuration.

Two details are worth noting. First, Fortinet’s own out-of-band feature is implemented with a VDOM, so the VDOM mechanism is the vendor’s answer when the goal is hard separation of a management port. Second, the same tip documents an HA issue, where a unit joining a cluster whose primary has the feature enabled can lose the management interface (fixed in 7.4.9, 7.6.4 and 8.0.0 per the article), and its workaround is to disable dedicated-mgmt and put mgmt1 in VRF 1 in the root VDOM with a static default route. So the vendor’s fallback for its VDOM-based approach is a VRF.

For a small branch with a management port and no tenants, this is often all you need, and it keeps you out of multi-VDOM mode. Note it covers inbound out-of-band access and its own default route. It does not by itself move FortiManager and logging traffic, so check which interface those services actually leave from.

Side by side

QuestionManagement VRFManagement VDOM
Local-out routing separationYes, per service, by pinning interface or vrf-select. Unpinned traffic follows the lowest VRF numberYes, management-class traffic originates in the management VDOM by default
Inbound admin reachabilityallowaccess, trusted hosts, local-in policyThe same. allowaccess is per interface
Administrative delegationNone. One global scopeYes. VDOM-scoped admins cannot reach global settings
Shares the WAN underlayYes, it is a routing instance on the same interfacesNo. An interface belongs to one VDOM, so traffic crosses an inter-VDOM link
SD-WAN configurationOneOne per VDOM, and management traffic transits the data VDOM’s rules
Overlay carriageCarry the VRF in the overlay, as in the MP-BGP designManagement traffic enters the data VDOM, then the overlay
FortiManager templatingOne device, VRF settings in the templateVDOM-aware templates and an extra axis of complexity
Licence and platformNoneVDOM count varies by model
Typical failureCross-VRF leak, VRF 0 asymmetry, a service that ignores pinningInter-VDOM policy gap, management VDOM loses its path, restrictions when moving the role
Migration costAdd a VRF, mostly liveEnable multi-VDOM, restructure, maintenance window
Audit storyWeak. The same people can change both planesStrong. Different people can own each plane

Which one, and when

One organisation, one network team, SD-WAN at hub and spoke scale: management VRF. The team that runs the data plane runs management, so a delegation boundary buys you nothing. The VRF rides the same underlay and overlay you already built, templates stay simple, and conversion is a live change. Strip allowaccess back, pin the services, and use a non-zero VRF for the management interface.

MSSP or multi-tenant hub: management VDOM. You need a boundary between your staff and each tenant, and between each tenant and the box’s own settings. This matches the MSSP hub discussion and the shared-services VDOM in the earlier VDOM post. A tenant should never be one set command away from changing where logs go.

Separate security or audit team that owns management: management VDOM, even in a single organisation. The justification is the audit story: the people who can change traffic flow are not the people who can change logging, authentication and central management. If the compliance requirement is written as separation of duties, a VRF cannot satisfy it.

Hubs where blast radius is the concern: lean VDOM, and only on hubs. A hub has more interfaces, more admins and more to protect, and it is usually a platform that can afford the VDOM licence and the extra design work.

Branch and spoke devices at scale: not a management VDOM. Per-VDOM SD-WAN, extra FortiManager complexity and VDOM limits on small platforms multiply across hundreds of sites for a benefit those sites rarely need. Use the VRF, or dedicated-mgmt where the model has a management port.

Small standalone site: neither. Dedicated management port, allowaccess limited to that port, trusted hosts on admins.

If I had to give one default, it is the VRF, with a clear trigger for moving to a management VDOM: the first time someone says “the other team must not be able to change this.” That sentence is a requirement for delegation, and delegation is what a VDOM sells.

What a lab should settle

Because this is a documentation-only post, here is what I would test before treating any of it as proven:

  1. Which services follow management-vdom on 7.6, in particular FortiManager (FGFM), FortiGuard updates and RADIUS or LDAP for admin login.
  2. Which local-out services offer vrf-select on 7.6.x, beyond the DNS example in the documentation, and which still fall back to the lowest-VRF rule.
  3. That a data-plane interface with https in allowaccess still accepts admin logins in multi-VDOM mode, and what a VDOM-scoped administrator can and cannot see from it.
  4. Behaviour of the RADIUS restriction when changing the management VDOM, on a current build.
  5. What happens to management reachability when the data VDOM’s SD-WAN path fails, in both designs.
  6. How FortiManager presents a management VDOM during device registration and policy install.

If the lab turns up anything that contradicts the above, I will follow up with a corrected post rather than quietly change this one.

Sources