FMG API From Scratch Part 8: Install, Preview First, Commit Second

Part 7 staged a change into the Device Database and proved it landed. Nothing has reached a live unit yet. This part is the step that does, and the aim is a tool that cannot install something you did not look at first.

The code for this part is in the fmg-api-from-scratch repo at tag part-08.

What install is

In the GUI, installing to a device is a wizard: pick the device, look at the preview, install. Behind it there are two separate things, and the API makes you do both:

  1. Preview. FortiManager works out what it would send to the device.
  2. Commit. It sends it.

The GUI sequence, from a browser capture of an install, is:

exec /securityconsole/package/cancel/install     clear any stale state
exec /securityconsole/install/device             flags: ["preview"]
exec /securityconsole/install/summary
exec /securityconsole/package/commit
exec /securityconsole/package/cancel/install     clean up

Two details matter before you write any code.

Numeric IDs, not names. The adom field is the ADOM’s numeric id (a number such as 169, not the ADOM’s name), and the scope is {"oid": <device>, "vdom_oid": <vdom>}. My first attempts used names, the way every other call in this series does. They failed with get install scripts error, err=-1, which says nothing useful, and I lost time to permissions and stale state before the scope shape turned out to be the cause. The numeric ids come from the Device Manager database:

def resolve(fmg, adom: str, device: str, vdom: str = "root") -> tuple[int, list[dict]]:
    """Numeric ADOM id and the install scope for one device's vdom."""
    adom_oid = fmg.get(f"/dvmdb/adom/{adom}", fields=["oid"])["oid"]
    rows = fmg.get(f"/dvmdb/adom/{adom}/device", fields=["name", "oid", "vdom"],
                   filter=["name", "==", device])
    ...

install/summary is optional. The GUI calls it, and for my least-privilege API account it returns -10. The commit works without it, so the tool does not call it.

The preview is two more calls

Here is the surprise. install/device with the preview flag finishes with state 4 and produces a summary, a list of member statuses. It does not produce the text the GUI shows when you click the eye icon. I called preview/result straight after it and got:

=== No preview result ===

which is true and no help. The diff comes from a second task:

exec /securityconsole/install/preview     {"adom": 169, "scope": [...], "flags": []}   -> task
GET  /task/task/<id>                       wait for state 4
exec /securityconsole/preview/result      {"adom": 169, "scope": [...]}

preview/result returns its answer as a JSON string inside data.message: a list of {name, oid, result}, where result is the CLI. For my change it parsed to:

=== Preview result ===
config system dns
    set domain "lab.test"
end

The diff is only what would change: one line, not the whole DNS block. The Ansible collection fortinet.fortimanager has modules for both calls (fmgr_securityconsole_install_preview and fmgr_securityconsole_preview_result), which is how I knew they were real and not something I had misread. preview/result refused the ADOM name and a device name on 7.6.6; the numeric ADOM and the numeric scope worked. There is also a json flag on these calls, but the release notes list it as a known bug, so I left it alone.

Parsing it has one trap of its own. “No preview result” is not an error and not an empty list. It is a successful reply whose text says there is nothing, so the tool turns it into an empty string and treats empty as “nothing to approve”:

def parse_preview(message: str) -> str:
    entries = json.loads(message)
    texts = [e.get("result", "") for e in entries if isinstance(e, dict)]
    texts = [t for t in texts if t.strip() and not t.strip().startswith(NO_RESULT)]
    return "\n".join(texts)

The guard: install only what you previewed

It is easy to build preview and install as two commands that happen to be run in order. That protects nothing: a script could stage something else between them, or a colleague could. I wanted install to prove that what it is about to send is what I read.

So preview records a hash of the diff, and install rebuilds the diff in its own session, hashes it, and compares:

def install(fmg, adom, device, state_dir, comment="fmg_push install", now=None) -> dict:
    approved = check_marker(state_dir, device, now)          # refuses: no preview, or too old
    adom_oid, scope = resolve(fmg, adom, device)
    try:
        diff = build_preview(fmg, adom_oid, scope, comment)
        if not diff:
            raise InstallRefused("the preview is empty: nothing to install")
        if diff_hash(diff) != approved:
            raise InstallRefused("the diff has changed since you previewed it; "
                                 "run preview again and read it")
        commit = _task(fmg, "/securityconsole/package/commit", {"adom": adom_oid, "scope": scope})
        status = wait_for_task(fmg, commit, timeout=300)
    finally:
        cancel(fmg, adom_oid)
    marker_path(state_dir, device).unlink(missing_ok=True)
    return {"device": device, "task": commit, "ok": succeeded(status), "sha256": approved}

Four rules sit in there:

  • No marker, no install. An approval has to exist. The marker is only written when there is a diff to approve.
  • Approvals expire. After 30 minutes the marker is refused and you preview again.
  • The hash is of the fresh diff, built in the same session as the commit, so it is a check on what is about to be sent and not on what was sent last time.
  • A marker is used once. A second install needs a second preview.

The tool cancels the pending install state in a finally, whether the commit went ahead, was refused, or failed. Dry run is the default: install without --apply sends nothing at all, which the tests prove by checking that a fake FortiManager received no calls.

The marker holds the device name, the hash and a time. It does not hold the diff. The diff is printed for you to read, with a reminder that it is the CLI that would be sent, secrets included.

Running it

This is a real run against a lab spoke, with its name replaced. First the change, staged (Part 7), then:

$ python fmg_push.py preview --device <spoke-1>
(this diff is the CLI that install would send, secrets included: treat it as sensitive)
=== Preview result ===
config system dns
    set domain "lab.test"
end

sha256 18091cad534d. Read it, then: install --device <spoke-1> --apply

$ python fmg_push.py install --device <spoke-1>
Dry run. Would install to <spoke-1> if the diff still matches your last preview. Add --apply to do it.

$ python fmg_push.py install --device <spoke-1> --apply
Install task 143 succeeded.

Then I put it back. I staged lab.local, and before previewing I tried to install anyway:

$ python fmg_push.py install --device <spoke-1> --apply
Not installed: no preview on record for <spoke-1>; run preview first

That is the guard doing its job, on a change I had staged and fully intended to install. After preview (hash 2773b3121744, one line, set domain "lab.local"), the install ran as task 149. Afterwards the device showed Synchronized in the GUI.

One case I tested with the fake FortiManager and not on the lab: a diff that changes between preview and install. That raises InstallRefused and never calls commit. I did not engineer that on the real box, so treat that one as unit-tested, not field-tested.

The preview is not the log

Part 7 ended on the point that the script log shows what the script ran. The install preview shows something different: what FortiManager will send, which is the difference between the Device Database and the unit. They can disagree. The log can say a script succeeded while the preview shows nothing, because the device already had the value; or the preview can show changes the script never made, because the Device Database had drifted from the device for another reason. Check the real install preview before you install. The guard in this part makes that the default, because the only diff it will let you install is the preview diff.

The tests

Thirteen tests cover the install module and three cover the command line. They use a fake FortiManager that records every call, so they assert on what was sent:

  • The preview chain runs in order, with the numeric ADOM and scope.
  • preview never calls commit and always cancels, even when a task fails.
  • “No preview result” parses to empty, and an empty diff writes no marker.
  • No marker, an expired marker and a changed diff each refuse the install, and none of them reach commit.
  • A matching diff commits exactly once and consumes the marker.
  • A failed commit is reported as a failure.
  • install without --apply sends nothing.

Checklist

  • Install is preview then commit. Build the preview and read it
  • ADOM and device are numeric ids here. Names fail with an error that does not say why
  • install/device with preview gives a summary. The diff needs install/preview and preview/result
  • “No preview result” is a successful reply that means nothing to show
  • Hash the diff at preview, rebuild it at install, and refuse if the hashes differ
  • Approvals expire and are used once
  • Always cancel the pending install state, whatever happened
  • Dry run is the default
  • The preview is the truth about what will be sent. The script log is not

What comes next

Part 9 puts it together: one command line for push, stage, log, preview and install, with an audit trail that records hashes and never content.