Ceph

Install

Use proxmox WebUI, no special options needed

Pool setup

Setup osds

Create a pool, set min size to 2 ( min size and size are nb of hosts )
After it was created, set it to 1. This is not recommended, but for now we only have 1 host operational ( and with enough storage )

Remove

systemctl stop ceph-mon.target
systemctl stop ceph-mgr.target
systemctl stop ceph-mds.target
systemctl stop ceph-osd.target
rm -rf /etc/systemd/system/ceph*
killall -9 ceph-mon ceph-mgr ceph-mds
rm -rf /var/lib/ceph/mon/  /var/lib/ceph/mgr/  /var/lib/ceph/mds/
pveceph purge
apt purge ceph-mon ceph-osd ceph-mgr ceph-mds
apt purge ceph-base ceph-mgr-modules-core
rm -rf /etc/ceph/*
rm -rf /etc/pve/ceph.conf
rm -rf /etc/pve/priv/ceph.*

These HAVE to be run on each nodes

Don't forget to restart your node after running these commands !