Proxmox Deep Dive Part 7: Clustering and Quorum

Part 6 covered the VM-versus-container decision. This part covers what changes the moment a homelab grows from one Proxmox node to more than one.

Corosync and pmxcfs, working together.

Part 1 mentioned that /etc/pve is actually pmxcfs, a FUSE filesystem that keeps configuration synchronized across every node in a cluster. Corosync is the underlying layer that makes that synchronization possible: it’s a group communication protocol handling cluster membership, node health detection, and message passing between nodes. Once corosync is up, pmxcfs uses it to propagate every write to /etc/pve to every other node in real time, which is why the web GUI shows the identical cluster-wide view no matter which node’s IP address is used to reach it.

Quorum: the mechanism that prevents a split cluster from making conflicting decisions.

Quorum is a voting system: each node gets one vote by default, and the cluster only considers itself “quorate” (authoritative, safe to act) when more than half of the total possible votes are present and reachable. The reason this matters isn’t abstract. If a network partition splits a 3-node cluster into a 2-node side and a 1-node side, only the 2-node side has quorum (2 of 3 votes, a majority), so only that side is allowed to make decisions like starting HA-managed VMs. The 1-node side, unable to reach a majority, refuses to act, specifically to prevent both sides from independently deciding to start the same VM and ending up with two copies of it running against the same disk.

Why two nodes is the genuinely awkward case.

A 2-node cluster has exactly 2 votes. If the two nodes lose contact with each other, neither one holds a majority (1 of 2 is not more than half), so neither considers itself quorate, and neither will act, even if one of them is perfectly healthy and the failure is purely a network split rather than an actual node failure. This is precisely the scenario QDevice exists to fix. A QDevice is an external, third-party arbitrator daemon, running on a separate machine outside the cluster, that casts a tie-breaking vote. With a QDevice added to a 2-node cluster, a network partition leaves the side that can still reach the QDevice with a majority (2 of 3, its own vote plus the QDevice’s), while the other side, unable to reach the QDevice either, correctly refuses to act. A 2+1 setup (two real nodes plus a QDevice) gets genuine split-brain protection without needing a third full Proxmox node purely to hold a vote, useful specifically for a homelab that has hardware for two nodes but not a third.

Live migration mechanics.

Moving a running VM between nodes without downtime depends directly on the storage decision from Part 2. With shared storage, live migration transfers only the VM’s running memory state and CPU context to the destination node, the disk itself never moves, since both nodes already see the same storage. With local storage only, Proxmox can still perform a live migration, but it has to stream the entire virtual disk’s contents to the destination node during the migration, which takes proportionally longer and consumes real bandwidth for large disks. ZFS replication, covered briefly in Part 8, offers a middle ground: periodic replication of a local ZFS dataset to another node’s local ZFS pool, so migration only has to transfer the delta since the last replication run rather than the entire disk.

Part 8 builds directly on quorum: High Availability, which uses the same cluster membership and voting mechanics to actually restart a VM automatically on a surviving node after a failure.