FMG API From Scratch Part 2: ADOM Database, Device Database and the Live Device

In Part 1 the argument was that writing to FortiManager and changing a FortiGate are two different operations. This part explains why, because everything else in the series rests on it. I got it wrong for weeks, so this is the post I wish I’d had at the start.

There are no API calls to run yet. There’s one thing to click in the GUI at the end, and a lot of diagrams.

Three places a configuration lives

For any managed FortiGate there are three copies of its configuration in play, and mixing them up is the single most common way to confuse yourself.

Where it livesWhat it holdsWho writes to it
ADOM databaseFortiManagerEverything shared in the ADOM: policy objects, policy packages, provisioning templates, AP profiles, FortiSwitch and FortiExtender templatesYou, in the GUI or API
Device databaseFortiManager, one per deviceA complete configuration file for that one FortiGate, as FortiManager believes it should beFortiManager, when it copies from the ADOM database. You, when you edit the device directly or run a script against it
The live deviceThe FortiGateThe running configurationFortiManager’s installer, or anyone with a console

The 7.6.6 Administration Guide describes the first two in its Key Concepts chapter:

ADOM Database: The FortiManager’s ADOM Database includes all ADOM objects including policy objects, provisioning templates, AP Profiles, FortiSwitch templates, and FortiExtender templates.

Device (FortiGate) Database: The FortiManager’s Device (FortiGate) Database has complete configuration files for each FortiGate that is managed by the FortiManager.

The word to hold on to is complete. The Device database isn’t a scratch pad of pending changes. It’s FortiManager’s full picture of the box.

What an ADOM actually is

An administrative domain is FortiManager’s unit of separation. In the guide’s words, ADOMs let administrators manage only the devices they’re assigned to, and an administrator account can be tied to one or more ADOMs or denied specific ones. Super_User accounts can see all of them.

A few facts worth knowing before you write code against them:

  • ADOMs are disabled by default. With them off you have a single root ADOM. Turning them on is a Super_User job.
  • ADOMs have types. Fabric, FortiGate, FortiCarrier and others. The type decides which devices can live in it.
  • The ADOM name goes in your URLs. You’ll see /dvmdb/adom/<adom>/device and /pm/config/adom/<adom>/obj/... throughout the series. An account scoped to one ADOM simply can’t see the others, which is what makes it a decent security boundary for an automation account (Part 4).
  • Workspace mode lets administrators lock ADOMs, devices or policy packages before editing. It can be set per ADOM. My lab ADOM runs in normal mode, so no explicit lock is needed. If yours has workspace mode enabled, check how your ADOM handles locking before you let a script write to it.

For more on the GUI side, my NSE 5 write-up on ADOMs and workspace mode and revision control cover it.

How an install really flows

Here’s the sequence from the guide, drawn as a picture:

   YOU (GUI or API)
        |
        |  write objects, templates, scripts, variables
        v
+----------------------+
|     ADOM DATABASE    |   shared objects, provisioning templates,
|  (per ADOM)          |   policy packages
+----------------------+
        |
        |  1. COPY   the parts that apply to this device
        |            ("VDOM copy operation")
        v
+----------------------+
|    DEVICE DATABASE   |   a complete config file for this one device
|  (per device)        |
+----------------------+
        |
        |  2. RETRIEVE the live config, DIFF it against the
        |             Device DB, install only the difference
        v
+----------------------+
|     LIVE FORTIGATE   |
+----------------------+

And the two steps in the guide’s own words (Key Concepts, pages 57 to 58):

  1. FortiManager copies the ADOM objects related to the change from the ADOM database to the Device database for the target FortiGate. The guide gives an example: each command line in a CLI template is applied to the configuration file stored in the Device database.
  2. FortiManager retrieves the current configuration from the real FortiGate, compares it to the newly updated configuration in the Device database, and installs the changes identified in the diff.

That note about the diff, quoted again because it matters:

The diff between the old and new configuration is installed to the target FortiGate, but not the original content. Because of this behavior, some object details (for example, some command lines in a CLI template) are not directly pushed to the target FortiGate.

Practically, this means a command in your template that already matches the live device produces no install at all. Keep that one in your head. It’s the reason a broken pipeline can look like a working one.

The order things are applied

For a real device, FortiManager applies the ADOM-level pieces to the Device database in a fixed order:

  1. System template
  2. CLI template (pre-VDOM copy)
  3. Threat weight template
  4. IPsec tunnel template
  5. Static route template
  6. BGP template
  7. NSX-T service template
  8. SD-WAN template
  9. AP profile
  10. FortiSwitch template
  11. FortiExtender template
  12. Policy package
  13. CLI template (post-VDOM copy)

Order matters when two templates touch the same object. If your CLI template sets something the system template also sets, whether yours wins depends on whether it’s a pre- or post-copy template. There’s a related but separate mechanism for model devices (devices you’ve pre-built in FortiManager before the real box exists), which have an extra pre-run CLI/Jinja step. I won’t cover those here.

Where each kind of script can run

FortiManager also has scripts, which are separate from provisioning templates and are what my own push tool uses. The Device Manager scripts chapter states where each kind can run:

  • CLI scripts can run on the Device database, on a policy package or the ADOM database, or directly on the remote FortiGate over its CLI.
  • TCL scripts can only run directly on the remote FortiGate.
  • Jinja scripts can only run on the ADOM database.

The last one is worth reading slowly, and I’ll quote it exactly:

Jinja scripts can only be run on the ADOM Database.

Now hold that against the previous section. An install works by diffing the Device database against the live device. A Jinja script only ever runs against the ADOM database. What happens to a Jinja script whose job is to change one particular device?

I didn’t think that through. It’s the story of Part 6, and the answer isn’t an error message.

The API mirrors the databases

Once you see the three places, the URL space stops looking random. These are the prefixes my own script uses, grouped by what they touch:

URL prefixWhat it’s aboutExample from my script
/sys/...The FortiManager session itself/sys/login/user, /sys/logout
/dvmdb/...The device and script inventory: ADOMs, devices, groups, scripts and their execution logs/dvmdb/adom/<adom>/device, /dvmdb/adom/<adom>/script/execute
/pm/config/adom/<adom>/...Objects in the ADOM database/pm/config/adom/<adom>/obj/fmg/script, /pm/config/adom/<adom>/obj/fmg/variable
/pm/config/device/<device>/...The Device database for one device/pm/config/device/<device>/global/system/dns
/securityconsole/...Actions, such as installing/securityconsole/install/device, /securityconsole/package/commit
/task/task/<id>Status of long-running actionsPolled after an install or script run
/cli/global/...FortiManager’s own system configurationAdmin profiles, for example

Two of those rows are the same idea seen from opposite sides. pm/config/adom/... reads and writes the shared objects, while pm/config/device/... reads and writes one device’s complete configuration. Which one you point at decides which database you touch, and the API will happily accept a write to either.

I have not checked every row against Fortinet’s own reference, which needs a developer-network login. Many of these paths appear in the community How-To or the Ansible collection, but treat the whole table as what I observed working on 7.6.6, and confirm a path on your own version before relying on it.

Reading the Device database directly

The most useful debugging habit I picked up was reading the Device database itself, instead of trusting a log or a status message. It’s a plain get:

{
  "id": 2,
  "method": "get",
  "params": [{ "url": "/pm/config/device/<device>/global/system/dns" }],
  "session": "<session-token>"
}

If you’ve written a change and want to know whether it landed in the place an install will read from, this is the check. Here’s the real response from my lab, trimmed, with the device name replaced and the primary resolver swapped for a documentation address:

{
  "result": [
    {
      "data": {
        "domain": ["lab.local"],
        "interface-select-method": 0,
        "oid": 153,
        "primary": "192.0.2.53",
        "protocol": 1,
        "secondary": "8.8.8.8",
        "server-select-method": 1,
        "vrf-select": 0
      },
      "status": { "code": 0, "message": "OK" },
      "url": "/pm/config/device/<device>/global/system/dns"
    }
  ]
}

Two things to notice. This is the Device database’s copy of the DNS settings, not a live read from the FortiGate. And enumerations such as protocol and server-select-method come back as integers, not the words you’d see in the CLI. Part 3 goes through how to send this request properly, and what to check in the response.

Have a look for yourself

You can see the Device database and the JSON FortiManager would send for a change without writing any code:

  1. In FortiManager, go to Device Manager, pick a device, and open its configuration. That’s the Device database, and it’s what the guide means by “the device database for each managed device”.
  2. Open an editable page, for example DNS under Network, and change a value in the editor without saving it.
  3. At the bottom of the editor click Preview. You’ll see the JSON API request for the change, with a toggle to show modified fields only and a copy button. The CLI Script tab shows the equivalent CLI.
  4. Close the editor without applying.

That’s a documented feature in the 7.6.6 Administration Guide (Device Manager, “Preview the JSON request or CLI script for changes in the device database”), and it’s the fastest way I know to learn what a real request looks like. Notice the URL in the JSON. It will be one of the pm/config/device/... paths from the table above.

What went wrong for me

I’ll save the full story for Part 6, but the shape of it belongs here because it’s the reason this part exists.

I believed my pipeline worked because every stage produced a convincing result. The script ran, the execution log showed correctly rendered CLI, and an install reported success. What I hadn’t checked was the Device database. When I finally read it directly, the value I thought I’d written wasn’t there. The install had been reporting success with “No cmds to be installed”, because the diff was empty, and it only ever looked like it had worked when the live device already happened to match what I was trying to push.

Nothing had errored at any point. The Device database was simply never written.

The rule I took from it: after any write, read back the database an install will use, not the log that describes the write.

Rules of thumb

  • Know which database a call touches before you send it. pm/config/adom/... is shared objects, pm/config/device/... is one device.
  • “Install succeeded” means the diff was applied, and an empty diff also succeeds.
  • A command that already matches the live device won’t be installed, so a successful install proves less than you’d hope.
  • Read the Device database directly whenever you want proof that a write landed.

If you use Jinja with FortiManager, my two template engines post is a good companion to Part 6, and the device registration write-up covers getting a device into an ADOM in the first place.

What comes next

Part 3 is the first API call. We’ll authenticate, read something, work out what a “successful” response actually means, and turn it into a small Python client class that every later part builds on.

Sources: