FMG API From Scratch Part 5: Discovery, or Finding What the Documentation Doesn't Tell You

Part 4 fenced the account. Now it has to find its way around. The FortiManager JSON API is large, the reference documents endpoints one at a time, and nothing tells you which of them your version, your ADOM and your account can actually use. This part is about surveying that ground before you build on it.

The code for this part is in the fmg-api-from-scratch repo at tag part-05.

Three sources, in order of trust

There are three ways to learn what an endpoint is:

Source	What it gives you	What it can’t tell you
The reference	The documented URL, its parameters, its types	Whether your version, ADOM and account can use it
The GUI	The exact request FortiManager itself sends for an action you can see	Anything the GUI doesn’t do
The box	What a get really returns, for you, today	Why, or whether it is supported

I trust them in the reverse order to how I reach for them. The reference tells me where to look, the GUI shows me a working request, and only the box settles what is true. Anything that reaches a script should have been confirmed by the box.

Where the documentation actually is

Part 1 has the full map of sources, so I won’t repeat it. The short version of what matters here: Fortinet’s API Best Practices Guide covers sessions, tasks, locking and logging; the community How-To covers login, scripts, the scope shape and script logs; and the Ansible collection fortinet.fortimanager is a good index of endpoints, including the install preview calls. “The API is not documented” is not quite fair. It is documented in pieces.

None of the pieces tells you what happens in the cases that cost me time: an enumeration that stores the default for any string, a status of 0 that means “nothing here”, an endpoint that returns a stale answer. That is what the box is for.

The URL trees

Every call from Part 3 onward has a url in params. It looks like a path, and it behaves like a path in a set of separate trees. These are the ones I used, with what each is for (names replaced):

Prefix	What lives there
/sys/...	Actions and status of FortiManager itself: login, logout, status
/dvmdb/...	The device and ADOM database: which ADOMs exist, which devices are managed, script runs
/pm/config/adom/<adom>/obj/...	Objects in the ADOM database: addresses, scripts, and so on
/pm/config/device/<device>/...	The Device database copy of one device’s configuration
/pm/pkg/adom/<adom>	Policy packages
/securityconsole/...	Install actions: preview, commit, cancel (Part 8)
/task/task/<id>	Asynchronous task state
/cli/global/...	FortiManager’s own configuration (administrators, profiles, settings)

The second and third rows connect back to Part 2. Reading the same DNS setting through /pm/config/adom/... and /pm/config/device/... asks two different databases. Neither is the live device.

Survey the box without leaking it

Reading an unknown endpoint and printing the result is a mistake, for the reason at the end of Part 4: routine reads contain credential material. Discovery needs a different question. Not “what is in this?” but “what kind of thing is this?”

So fmg_discover.py reports only the shape of a response: whether it’s a list or an object, how many items, and the field names. Never the values.

def shape(data: Any) -> str:
    """Describe a response without quoting any value."""
    if isinstance(data, list):
        if not data:
            return "list, 0 items"
        first = data[0]
        keys = sorted(first) if isinstance(first, dict) else []
        return f"list, {len(data)} items; item keys: {', '.join(keys) or type(first).__name__}"
    if isinstance(data, dict):
        return f"object; keys: {', '.join(sorted(data))}"
    return f"scalar ({type(data).__name__})"

probe wraps a get and files the result under one of five outcomes:

Outcome	Meaning
data	Status OK and a data block came back
empty	Status OK but no data key at all: nothing to return for these arguments, or no such endpoint
denied	-11: no permission, or no such path
missing	-3: the object doesn’t exist
error	Anything else, with the code

The empty case is the one to respect, because it is ambiguous. A status of 0 doesn’t prove an endpoint is real, and a missing data key doesn’t prove it is fake. The client turns it into an error rather than an empty result, and the probe reports it as empty rather than as success.

The way to resolve it is to ask the same endpoint something you know has an answer. The probe can’t do that for you, so I do it by hand: ask one URL about several devices, and if some return data and one returns nothing, the URL is real and the empty one has nothing to show. (Part 7 covers a case where a real answer was still the wrong one.)

A subtlety in the ordering matters: FMGShapeError is a subclass of FMGError. The probe has to catch the subclass first, or empty would be swallowed by the general handler and reported as an error. There is a test that fails if the order is swapped.

Everything is read-only. The script cannot write, because it only calls get.

What my box said

This is the real survey from my lab, with names replaced and the field lists trimmed. The account is the narrowed one from Part 4.

Endpoint	Result
/sys/status	Object. Keys include version, build, hostname, serial number, licence status
/dvmdb/adom	List, 1 item: the account’s own ADOM
/dvmdb/adom/<adom>	Object: the same record
/dvmdb/adom/<adom>/device	List, 4 items, around 100 fields each
/dvmdb/device	List, 4 items: the same four
/dvmdb/adom/<adom>/script	List, 0 items
/pm/config/adom/<adom>/obj/fmg/script	List, 8 items
/pm/pkg/adom/<adom>	List, 2 items (policy packages)
/pm/config/adom/<adom>/obj/firewall/address	List, 18 items
/pm/config/device/<device>/global/system/dns	Object, 28 keys
/pm/config/device/<device>/global/system/interface	List, 27 items, roughly 190 fields each
/dvmdb/adom/<adom>/script/log/latest/device/<device>	Object: content, exec_time, log_id, script_name (but see below and Part 7)

Nothing came back denied, empty or missing for the device I happened to pick. That is a result about this box and this device: the point of the tool is that you run it on yours.

Things worth reading out of it

The word “script” names two different things. The /dvmdb/.../script path returned zero items. The object path /pm/config/.../obj/fmg/script returned eight, and those are the scripts I had made. A status of 0 with an empty list is a legitimate answer to a different question. It isn’t an error, and it isn’t the place my scripts live. If you had only tried the first path you’d have concluded there were none.

One account, one ADOM. /dvmdb/adom lists a single ADOM, which is Part 4’s where fence seen from the inside: the account can’t even list the others, let alone read them.

The device records contain secrets, and you can see it from the field names alone. The device list has fields called adm_pass, private_key, psk and tunnel_cookie among its hundred or so. That is why the survey prints names and never values, and why the redaction in Part 4 exists. A survey that printed values would have put credential material in my terminal history.

Responses are big. One device’s interface list is 27 interfaces with around 190 fields each. Ask for less, which brings us to fields and filter.

fields narrows a record, but not to exactly what you ask for. Asking the device list for just name returned five keys per device, not one; asking for the ADOM list’s name returned two. In both cases the extras were oid plus a few others (on devices, ha_slave, onboard_rule and vdom). On the interface list, asking for name cut a record of about 190 fields to 14, and the extras there were mostly names of nested tables such as ipv6 and egress-queues. So:

Request	Records	Keys per record
Devices, fields: ["name"]	4	5 (name, oid and three others)
Devices, fields: ["name","ip"]	4	6 (the same extras, plus ip)
ADOM list, fields: ["name"]	1	2 (name, oid)
Interfaces, fields: ["name"]	27	14 (name, oid and nested-table names)

The lesson is to measure the effect rather than assume it: the useful question is “did the record get smaller?”, not “did I get only what I asked for?”. The helper in the repo reports the key counts with and without the filter for exactly that reason. Part 4 noted that fields was ignored on /cli/global/system/admin/user, so this is worth testing endpoint by endpoint.

filter selects records, not columns. Filtering the interface list on name == wan1 returned one interface, but with all of its 155 fields. It shrinks the number of rows, and fields shrinks the width. Combine them when you need one column of one row.

The script log has several doors, and they are not interchangeable. /dvmdb/.../script/log/latest/device/<device> looks like the obvious one. It returns a record with the text of a run, when it ran, its script name and a log_id. I asked it about all four devices: three returned data, and FortiAnalyzer, which has never had a script run, returned status 0 with no data. That pair told me the endpoint was real and “no data” meant “no run recorded”. What it did not tell me was whether the record was the newest, and it wasn’t: a later check found it returning a run weeks old. The reliable path is the per-device summary plus the per-device output, and Part 7 explains how to find the right run from the task id.

The status output identifies the box. /sys/status includes the hostname and serial number as keys. The names are harmless. The values are exactly what should be redacted before this appears anywhere public, including in a screenshot.

Discovering a new endpoint safely

When you want to do something and don’t know the URL, the order that has worked for me:

1. Search the reference for the object or action, and note the documented URL. The Ansible collection’s module list is a quick index.

2. Do it once in the GUI with the browser’s network tab open. Find the request to /jsonrpc. Its body has the method and params[0].url that FortiManager itself used.

3. Copy just the body and pull out the method and URL. The repo has a small helper for this that ignores everything else, session token included:

   python examples/discover.py --parse captured.txt

4. Replay it as a get first. If the GUI action was a change, look at the same URL with a read before you ever send the write.

5. Read with verbose (Part 3) so enumerations come back as words, and compare with what the GUI shows.

6. Write it down with where you learned it: reference, GUI capture, or your own probe. Part 6 depends on knowing which paths are documented.

Capturing the GUI is not always easy. One of my exports of the install wizard came out with no requests in it at all, so check that the recording is running before you start the action, and that you are recording the window the action happens in. When a capture won’t cooperate, a read-only probe of the calls you suspect will often settle it. That is how I found the install preview calls in Part 8.

One honest limit: the capture parser is unit-tested on made-up input, not on a live browser export, so check what it extracts against the request you can see in the network tab before you trust it.

The rule under all of it: never replay a captured write to learn what it does. Find out what it does by reading the state it changes, before and after, from a copy of the thing.

The tests

The discovery helpers are unit-tested like everything else, and each test guards something that could quietly go wrong:

• shape never includes a value, only names and counts. The test passes a record containing recognisable secrets and asserts none of them appears in the output.
• A missing data key is reported as empty, not data.
• -11 and -3 map to denied and missing, and anything else to error with its code.
• The fields check reports the record width with and without the filter, and says so when nothing came back.
• The capture parser drops the session token, de-duplicates, and survives input that isn’t JSON.

Checklist

• Learn from the reference, confirm with the GUI, settle it on the box
• Know the trees: sys, dvmdb, ADOM objects, device database, packages, securityconsole, task, cli/global
• Print the shape of an unknown response, never its values
• Treat status 0 with no data as a question: nothing to return, or no such endpoint? Answer it by asking the same endpoint something that has an answer
• A response that exists is not a response that is current
• A different path with the same word in it is a different thing
• Ask for less with fields (columns) and filter (rows), and measure what each does on that endpoint
• Record where each endpoint came from
• Never replay a captured write to find out what it does

What comes next

We can now find things and read them. Part 6 starts changing them: creating scripts through the API, and the trap that catches Jinja scripts run against the wrong database.