Set up Ceph on a Proxmox cluster (hyper-converged storage)

Proxmox Ceph combines compute and storage on the same nodes. Every VM disk is replicated across servers, so a node can fail without losing data, and VMs can live-migrate and use HA without a separate storage array.

Before you start

• A Proxmox cluster with at least three nodes (cluster setup).
• Dedicated disks for Ceph on every node (SSD or NVMe recommended), with no RAID controller in RAID mode; use HBA/IT mode.
• A dedicated network for Ceph, ideally 10 Gbit/s or faster, separate from public traffic.
• Enough RAM: plan several GB per OSD on top of your VMs.

Background on Ceph concepts: Ceph storage for virtualisation.

Install Ceph

1. Select the first node → Ceph and click Install Ceph.
2. Pick the Ceph version offered and the repository (no-subscription if you have no subscription).
3. On the configuration step, set the public network and, if you have one, a separate cluster network for replication traffic.
4. Repeat the installation on every node.

Create monitors and managers

1. Ceph → Monitor: create a monitor on three nodes (an odd number keeps quorum).
2. Create a manager on at least two nodes for redundancy.

Create OSDs

1. On each node: Ceph → OSD → Create: OSD.
2. Select an empty disk (wipe old partitions first) and create.
3. Repeat for every Ceph disk on every node. Keep the number and size of OSDs similar across nodes.

Create a pool for VM disks

1. Ceph → Pools → Create.
2. Use size 3, min size 2: three copies, still writable with two.
3. Keep Add as Storage ticked; the pool appears as RBD storage for VM and container disks on all nodes.

Check health

ceph -s

Look for HEALTH_OK and all placement groups active+clean. The Ceph dashboard in the web UI shows the same.

Troubleshooting

• HEALTH_WARN clock skew: sync time on all nodes with NTP (chrony).
• Slow writes: Ceph on 1 Gbit/s or consumer SSDs without power-loss protection is slow; upgrade the network and disks.
• OSD down: check the disk with smartctl and replace it; Ceph rebalances automatically.

Frequently asked questions

Can I run Ceph on two nodes?

Not safely. Three nodes is the minimum for production.

Do I still need backups?

Yes. Replication protects against hardware failure, not deletion or corruption.

Ceph or ZFS replication?

Ceph gives shared storage with no data-loss window; ZFS replication is simpler for two or three nodes (ZFS replication).

Next steps

• High availability
• ZFS replication
• Storage types