Proxmox Deep Dive Part 9: Ceph Hyperconverged Storage

Part 8 named shared storage as one of HA’s real requirements. Ceph is Proxmox’s own native answer to that requirement: a distributed storage system built from the cluster’s own nodes, rather than an external NFS or iSCSI target.

Architecture, briefly.

Ceph is built from OSDs (Object Storage Daemons, one per physical disk contributing to the cluster), monitors (maintaining the cluster map and quorum, similar in spirit to corosync’s role for Proxmox clustering itself), and managers (handling metrics and additional cluster management functions). Data placement across OSDs is governed by CRUSH, an algorithm that calculates where each piece of data should live without needing a central lookup table, which is what lets Ceph scale without a single point of failure for data location itself. Proxmox integrates Ceph directly, installable and manageable through the same web GUI as everything else, or via the pveceph command-line tool for scripted setups.

The minimum node count is a real, non-negotiable floor.

Ceph’s default replication factor stores three copies of every object, which means a meaningful Ceph deployment needs at least three nodes contributing OSDs, not as a best practice but as a structural requirement of the default replication scheme. Single-node and two-node Ceph configurations are technically possible with adjusted replication settings, but they sacrifice the redundancy that’s the entire point of running Ceph in the first place, at which point the complexity Ceph adds isn’t buying back anything ZFS with local storage wouldn’t already provide more simply.

Version currency matters more with Ceph than with most other Proxmox subsystems.

Ceph ships versioned releases with names (Reef, Squid, and so on) rather than tracking Proxmox VE’s own version number, and upgrading between them has real prerequisites, moving from Reef (18.2.x) to Squid (19.2.x) requires Proxmox VE 8.2 or newer with pve-manager at 8.2.8 or newer and Ceph packages at 18.2.4-pve3 or newer before the upgrade path is even available. This is one of the few subsystems in this series where checking the current documented upgrade path before touching anything is genuinely necessary rather than a formality, an out-of-sequence Ceph upgrade can leave a cluster in a broken intermediate state.

Performance reality check against ZFS.

For a three-node homelab cluster built with SSDs, Ceph delivers genuinely solid performance and redundancy, and where true multi-node shared storage is the actual goal, it’s the native option that doesn’t require standing up a separate NAS. But Ceph’s network overhead (every write typically replicates across the network to two other nodes before it’s acknowledged) and operational complexity are real costs that ZFS with local storage plus periodic replication (Part 7) simply doesn’t carry. For a homelab that doesn’t specifically need live migration and HA to work across true shared storage, ZFS with replication delivers most of the practical resilience at a fraction of the hardware floor and operational overhead, one or two nodes, not a mandatory three.

When Ceph earns its complexity in a homelab, and when it doesn’t.

Worth it: three or more nodes are already available, HA and live migration across genuinely shared storage are an actual goal rather than a nice-to-have, and the operational overhead of learning and maintaining Ceph is accepted going in. Not worth it: fewer than three nodes, or the actual goal is “resilient storage on a couple of boxes” rather than “true hyperconverged shared storage,” in which case ZFS with replication or even a dedicated NFS/iSCSI target reaches the same practical outcome with meaningfully less to maintain.

Part 10 moves away from live storage entirely, to backup: vzdump, retention scheduling, and Proxmox Backup Server as the recommended companion product for anything beyond ad hoc snapshots.