Beyond PSK: PKI for Fortinet SD-WAN IPsec, Part 1 — The Architecture Decision

Beyond PSK: PKI for Fortinet SD-WAN IPsec, Part 1 — The Architecture Decision

Series scope. This is a three-part deep dive into running certificate-based IPsec on a Fortinet SD-WAN estate properly — not as a one-off migration, but as a PKI you can operate for a decade. Part 1 (this post) is the architecture decision: FortiManager-as-CA versus a dedicated external CA, SCEP versus EST, CRL versus OCSP, certificate lifetime philosophy, and what actually changes at the IKEv2 protocol level when you swap a shared secret for a signature. Part 2 stands up the PKI itself — CA hierarchy, certificate templates, EST front-end, and FortiManager’s (much smaller) supporting role. Part 3 is enrollment automation at scale and the PSK-to-certificate cutover on the FortiGates themselves.

Prior art. If you’ve read Jinja Orchestrator Part 3, you’ve already seen one path through this problem — FortiManager acting as its own CA, which is the fastest way off PSK if FMG is the only PKI-shaped thing you own. This series doesn’t replace that post; it sits next to it and answers the question that post deliberately left open: what do you do when “fastest” and “right for the next ten years” aren’t the same answer?

The question hiding inside “switch to certificates”

Every FortiGate SD-WAN estate eventually has the PSK conversation. Someone reads an audit finding, or a pen test report, or just stares too long at set psksecret S3cr3t! in a Project Template and asks the obvious question: why does every spoke in the overlay carry the same plaintext shared secret, and what happens when one of them grows legs?

The answer — move to certificate-based IKEv2 authentication — is universally agreed on. What’s not universally agreed on, and what most migration writeups skip past, is the question that determines whether the resulting system is a genuine improvement or a more expensive way of having the same operational problems: who is your CA, and how do certificates get onto devices without a human in the loop forever?

That’s not a Fortinet question. It’s a PKI question wearing a Fortinet costume. Get it right and certificate-based IPsec is a system that runs itself — devices enroll on first contact, renew themselves quietly months before expiry, and a stolen spoke is a CRL/OCSP entry away from being cryptographically irrelevant. Get it wrong and you’ve built a more elaborate PSK: a shared trust anchor everyone holds, a manual enrollment process that doesn’t scale past the size of estate where you stopped counting devices on your fingers, and a renewal cliff waiting for the day everyone’s certificates expire in the same maintenance window because they were all issued from the same CLI loop on the same Tuesday.

So before any FortiOS CLI appears in this series — and it will, starting in Part 2 — we need to make the architecture decision properly. That means being honest about what FortiManager’s built-in CA actually is, what a dedicated external CA actually costs, and which one is the right foundation for an enterprise-grade deployment rather than the right foundation for a lab or a 20-site proof of concept.

Option A: FortiManager as the CA

Part 3 of the Jinja Orchestrator series walked through how this actually works in practice, so I’ll summarize rather than re-derive: FMG can generate a local root CA, mint a per-device certificate signed by that root for every managed FortiGate, push the certificate and the CA trust anchor down over FGFM, publish a CRL that devices fetch over HTTPS, and offer scheduled or on-demand renewal.

That is a genuinely useful feature, and for the right estate it is the correct answer. Here’s the honest breakdown of where it’s strong and where it stops being a CA and starts being a certificate-shaped extension of the device-management plane.

Where it’s strong:

• Zero new infrastructure. If you already run FMG to manage the estate, the CA function is a checkbox away. No new servers, no new patching surface, no new thing to put on the DR runbook.
• Enrollment rides the existing management channel. Certificates and trust anchors arrive over FGFM, the same authenticated channel that already pushes config. There’s no separate enrollment protocol to stand up or secure.
• Operationally legible for a single team. One pane of glass for device management and certificate management. For a single-tenant shop with a few hundred edges and one ops team, that’s a real reduction in moving parts, not a corner cut.

Where it stops being enough:

• No EST, no real SCEP-from-the-device-side story. FMG enrolls devices by pushing — it generates the cert and ships it down. It does not run an EST or SCEP server that a FortiGate calls out to on its own initiative. That sounds like a nuance, but it’s the single biggest constraint on the architecture: it means FMG must always be the active party in enrollment, which means a device that’s been re-imaged, factory-reset, or shipped cold to a new site cannot self-enroll — it waits for FMG to notice it and push. At ten sites that’s a non-issue. At a thousand, it’s a queue.
• CRL only — no OCSP. Revocation reaction time is bound to whatever CRL refresh interval you configure, full stop. There’s no responder to query for a near-real-time answer.
• One root per FMG/ADOM. That’s fine for a single tenant. It is not fine for an MSSP managing cryptographically distinct customer estates from one FMG — there’s no clean way to give Customer A and Customer B their own CA hierarchies with proper separation while still running them from the same management plane. Part 2 of the Hub Placement series covered the broader version of this MSSP-shaped problem; the CA question is the same shape again.
• The root lives where the device manager lives. Compromise FMG and you haven’t just lost device management — you’ve lost the thing that vouches for every device’s identity. A dedicated CA, properly separated, means a compromised management plane doesn’t automatically mean a compromised trust anchor.
• It’s an island. The FMG CA signs certificates for IPsec on FortiGates. It does not, and structurally cannot, become the CA for your RADIUS EAP-TLS deployment, your admin-GUI certificates, your syslog mTLS, or anything else that wants a certificate and isn’t a FortiGate phase1. You end up running two PKIs — one for Fortinet, one for everything else — which is the exact kind of duplicated, divergent infrastructure that audits exist to find.

None of this makes FMG-as-CA wrong. It makes it scoped: the right tool for “I need to get off PSK with what I already have, on an estate small enough that push-based enrollment doesn’t become a queue, and I don’t need this CA to do anything beyond sign FortiGate IPsec certs.” That’s a real, common, entirely defensible position. It is just not the enterprise-grade position, and a post that pretends otherwise would be doing you a disservice.

Option B: a dedicated external CA

The alternative is to treat certificate issuance as what it actually is — a PKI problem — and solve it with PKI tooling: a Microsoft AD CS hierarchy (root + issuing CA, NDES for SCEP, an EST proxy in front if you want EST), a HashiCorp Vault PKI secrets engine (intermediate CA mounts, role-based issuance, native API-driven automation), or an ACME/EST-native CA like Smallstep step-ca. The specific product matters less than the shape: an offline or tightly-controlled root, one or more online issuing CAs, and a protocol-driven enrollment front-end that devices call out to.

What you get that FMG-as-CA structurally cannot give you:

• EST support, natively. This is the headline. AD CS can sit behind an EST proxy (or, depending on your Windows Server version and tooling, expose EST-compatible enrollment more directly); Vault’s PKI engine pairs cleanly with EST front-ends; step-ca speaks EST out of the box. That turns enrollment from “FMG notices and pushes” into “the device calls home, authenticates with what it already has, and gets what it needs” — which is the actual definition of automation, and it’s the protocol FortiOS itself has supported as a client since 7.4.3.
• OCSP. A responder you can query for a near-real-time revocation answer, instead of waiting out a CRL interval.
• Real multi-tenancy. Intermediate CAs per tenant, per region, per business unit — proper cryptographic separation with one root anchoring all of it, which is exactly what an MSSP (or a large enough enterprise with distinct business units) needs and FMG-as-CA cannot provide.
• One PKI for the whole estate, not one for Fortinet and one for everything else. The same issuing CA that signs IPsec end-entity certificates can sign your RADIUS server certs, your admin-GUI certs, your internal mTLS — all under one auditable hierarchy, one renewal philosophy, one root rotation plan.
• Proper root/issuing separation. The root can live offline — generated once, used to sign issuing CAs, then locked in a safe (literally or figuratively, an HSM in a vault either way). Compromise of an issuing CA is a survivable, recoverable event. Compromise of the root is not, and FMG-as-CA gives you no way to keep the root cold.

What it costs you, honestly:

• You are now operating a CA. That’s key protection (HSM or HSM-equivalent), patching the issuing servers, engineering CRL/OCSP availability so it’s reachable from every site — including spokes behind NAT’d, firewalled, sometimes-flaky underlays — and building the renewal-automation glue (EST front-end config, certificate templates/roles, auto-enrollment policy) that makes the whole thing actually hands-off rather than just differently manual.
• More moving parts means more things that can fail in ways that take down tunnel establishment estate-wide. A PKI outage with a well-designed cert lifetime policy is an inconvenience. A PKI outage with a badly-designed one is every site’s IKE_AUTH failing in the same hour.

That’s not a small list. It’s the actual cost of “enterprise-grade,” and pretending it’s free would be the same mistake as pretending FMG-as-CA can do things it can’t.

The call

For an enterprise-grade Fortinet SD-WAN deployment, the dedicated external CA is the right architecture — specifically, an issuing tier (AD CS or Vault PKI) fronted by an EST endpoint, with FortiManager doing what it’s actually good at: pushing the configuration that points devices at the PKI — trust anchors, CRL/OCSP server config, EST client parameters — rather than trying to be the PKI itself.

That’s not a knock on FMG-as-CA. It’s the right call for a smaller or single-tenant estate, and if that’s where you are, Part 3 of the Jinja Orchestrator series is your migration guide — go read it, you’re done, come back to this series if you ever outgrow it. The rest of this series is for the estate that needs EST’s pull-based automation, OCSP’s revocation latency, multi-tenant separation, or a PKI that serves more than one product family. If any of those are true for you, keep reading — Parts 2 and 3 build exactly that.

The deciding factor, if you only remember one thing from this section: FMG-as-CA is push-based and SCEP-shaped; an external CA with an EST front-end is pull-based and self-service-shaped. Push-based enrollment is fine until the size of your estate makes “FMG notices and acts” the bottleneck. Pull-based enrollment scales because the device does the work the moment it has the means to.

SCEP vs. EST — the fork that decides everything downstream

This is worth its own section because the entire “automation” promise of certificate-based IPsec lives or dies on this distinction, and it’s frequently glossed over as “two enrollment protocols, pick one.”

SCEP (Simple Certificate Enrollment Protocol, formalized as RFC 8894) is the legacy default. It runs over plain HTTP, authenticates enrollment requests with a shared challenge password (a one-time or reusable secret the device presents to prove it’s allowed to enroll), and has no native concept of “renew using what you already have.” Practically, that means:

• The challenge password is itself a shared secret you now have to distribute and protect — which, if you squint, is uncomfortably close to the PSK problem you’re trying to solve.
• Renewal is “do SCEP enrollment again,” which means the device needs another valid challenge, which means someone (or some system) has to keep generating and distributing them.
• Crypto-agility is weak — SCEP’s message format and trust model were designed in an era with different assumptions about acceptable algorithms and transport security.

EST (Enrollment over Secure Transport, RFC 7030) was designed specifically to fix this. It runs over TLS (so the transport is secure by construction, not by hope), and — this is the part that matters — it supports certificate-based re-enrollment: a device that already holds a valid certificate can authenticate to the EST server with that certificate, via mutual TLS, and receive its next one before the current one expires. No shared secret. No challenge password to rotate and distribute. The device’s existing cryptographic identity is the credential that gets it its next cryptographic identity.

That’s the actual automation story. It’s also precisely the protocol FortiOS gained native client support for — EST support landed in FortiOS 7.4.3 for CSR generation, automatic renewal, and manual regeneration. If your FortiOS estate is on 7.4.3 or later — and if you’re running a current SD-WAN deployment, it should be — EST is not a future-proofing bet. It’s available now, and it’s the protocol that turns “certificate lifecycle” from a recurring task into a property of the system.

The one wrinkle: EST’s elegant “authenticate with your current cert” model needs a first cert to bootstrap from. That bootstrap step — typically a one-time enrollment using an out-of-band credential or a short-lived SCEP-style challenge — is the one place a human (or a provisioning system) still has to be involved. We’ll design that bootstrap flow properly in Part 3, because doing it sloppily is how you end up with “automated renewal” sitting on top of a manual, unscalable, and quietly-load-bearing first-enrollment process.

CRL vs. OCSP — and the chicken-and-egg problem unique to overlays

Revocation checking has the same legacy/modern split as enrollment, and it has an extra wrinkle that’s specific to SD-WAN: the thing you’re checking revocation status to protect is often the same thing you need up to reach the revocation-status server.

CRL (Certificate Revocation List) is a periodically-published, signed list of revoked certificate serial numbers that devices fetch from a distribution point. It’s simple, it caches well, and it’s universally supported — but your revocation reaction time is bound directly to your refresh interval. Revoke a stolen spoke’s certificate now, and every other device in the estate honors that revocation the next time it fetches the CRL. Set the interval to an hour, and your worst-case exposure window after a revocation is an hour. That’s the trade you’re explicitly making, and for most estates it’s a perfectly reasonable one.

OCSP (Online Certificate Status Protocol) replaces “fetch a list and check it locally” with “ask a responder about this one specific certificate, right now.” Properly deployed, your revocation reaction time approaches real-time. The cost is that you now need a responder that’s reachable, available, and fast — from every device, including spokes sitting behind NAT’d, sometimes-marginal underlay connections, possibly without a clean path to the responder that doesn’t itself depend on the overlay being up.

Which brings us to the chicken-and-egg problem: a device needs the IPsec overlay up to reach a CRL distribution point or OCSP responder that lives on the other side of that overlay — but the overlay needs a validated certificate to come up in the first place. Part 3 of the Jinja Orchestrator series flagged the sharp edge of this (an expired cert means the device can’t reach the CA to renew, which means it stays expired — a recovery problem). The revocation-checking version is the same shape but runs the other direction: a device whose cert is fine still needs to validate its peer’s cert, which means reaching a CRL/OCSP endpoint, which — depending on your topology — may or may not be reachable independent of the tunnel currently being negotiated.

The practical answer, and the one we’ll design around in Part 2, is to make the distribution points reachable over the underlay, not the overlay — a publicly-reachable (or at least underlay-reachable) CRL/OCSP endpoint that doesn’t have a circular dependency on the thing it’s helping to authenticate. Get this placement decision wrong and you’ve built a system that works perfectly in the lab and falls over in exactly the scenario it exists to handle: a device coming up cold.

Certificate lifetimes — and why “twenty years and forget it” is the wrong instinct now

PSK-era thinking treats credential lifetime as a cost to be minimized — rotate as rarely as possible, because rotation is a coordinated, painful, all-or-nothing event. That instinct, carried unchanged into certificate lifetime policy, produces exactly the wrong design: long-lived leaf certificates that nobody wants to touch, “managed” with the same dread as a PSK rotation, defeating half the point of moving to certificates in the first place.

The right instinct flips once renewal is actually automated. With EST-driven auto-renewal, a short leaf lifetime isn’t an operational burden — it’s a feature. It bounds your exposure if a key is ever silently compromised without triggering a revocation event (the cert ages out on its own), it forces the renewal automation to be exercised constantly rather than once a decade when nobody remembers how it works, and it makes “has renewal been silently broken for six months” a question your monitoring answers in days, not years.

The hierarchy, then, looks like this:

• Root CA: long-lived, generated once, ideally kept offline. This is the one place “decades, and write the expiry on a calendar” is the correct answer — root rotation is a full trust-anchor redistribution to every device in the estate, and you do not want to do it more often than strictly necessary.
• Issuing CA: medium-lived — long enough that issuing-CA rotation is a planned, infrequent project rather than a fire drill, short enough that a compromised issuing CA has a natural expiry backstop.
• Leaf (end-entity / IPsec) certificates: short-lived, renewed automatically and continuously. This is where the EST automation does its real work, and it’s the layer where “short” stops being a liability and starts being the thing that makes the rest of the system trustworthy.

We’ll turn this philosophy into actual template/role parameters — key usage, EKU, validity windows — in Part 2.

What actually changes at the IKEv2 layer

It’s worth being precise about what “switch from PSK to certificates” changes at the protocol level, because the answer is narrower — and more interesting — than “everything.”

In IKEv2, the IKE_AUTH exchange is where each peer proves its identity to the other. With authmethod psk, that proof is a MAC computed over the negotiated material using the shared secret — both sides know the same secret, so both sides can compute and verify the same MAC. There’s no cryptographic binding between that specific device and the secret; there’s only “knows the secret or doesn’t.”

With authmethod signature, the AUTH payload becomes a digital signature over the signed octets, computed with the device’s private key. The receiving peer validates that signature using the public key from the presented certificate — and then, critically, validates the certificate itself by chaining it back through the issuing CA to a trusted root. That second step is the entire point: it’s not “did you sign with a key,” it’s “did you sign with the key that belongs to the certificate that a CA I trust says belongs to you, specifically.”

This is also where peertype and the user peer / user peergrp constructs earn their keep — and why they’re not just cosmetic filtering:

• peertype any accepts any certificate that chains to a trusted CA. That’s a meaningful improvement over PSK (identity is now cryptographically bound to a specific keypair) but it’s coarse — any valid cert from any device that CA has ever signed is acceptable.
• peertype peer plus a user peer object narrows that to certificates matching specific subject criteria — typically CN or issuer-subject filters. This is where subject naming conventions stop being a cosmetic choice (Part 3 of the Jinja Orchestrator series called this out as “per-device cert names are pointless visual variation”) and become a genuine access-control mechanism: if your CA issues certificates to things that aren’t your FortiGates — and in a shared-PKI, multi-purpose CA model, it will — peertype peer filtering is what stops “any cert this CA ever signed” from being a back door into your overlay.

That last point is, not coincidentally, the strongest argument for getting your subject-naming and issuance-policy conventions right at the CA level — which is exactly where Part 2 starts.

Where we’re going

Part 2 builds the thing this post argued for: a CA hierarchy with a properly-separated root and issuing tier, certificate templates/roles scoped specifically to IPsec end-entity use, an EST front-end FortiGates can actually call out to, CRL/OCSP distribution points placed where the chicken-and-egg problem can’t reach them — and FortiManager’s role redrawn to match what it’s actually for in this architecture: pushing the configuration that points devices at all of the above, not minting the certificates itself.

Part 3 is where the FortiGates start talking to that PKI: enrollment automation at scale, the bootstrap problem EST can’t solve on its own, monitoring renewal health before it becomes an outage, and the IPsec template migration itself — building on, and explicitly diffing against, the FMG-as-CA version of that migration that Jinja Orchestrator Part 3 already walked.