Ceph

Install
Use proxmox WebUI, no special options needed

Pool setup
Setup osds 

 Create a pool, set min size to 2 ( min size and size are nb of hosts ) After it was created, set it to 1. This is not recommended, but for now we only have 1 host operational ( and with enough storage )

Remove
systemctl stop ceph-mon.target

systemctl stop ceph-mgr.target

systemctl stop ceph-mds.target

systemctl stop ceph-osd.target

rm -rf /etc/systemd/system/ceph*

killall -9 ceph-mon ceph-mgr ceph-mds

rm -rf /var/lib/ceph/mon/ /var/lib/ceph/mgr/ /var/lib/ceph/mds/

pveceph purge

apt purge ceph-mon ceph-osd ceph-mgr ceph-mds

apt purge ceph-base ceph-mgr-modules-core

rm -rf /etc/ceph/*

rm -rf /etc/pve/ceph.conf

rm -rf /etc/pve/priv/ceph.* 

 These HAVE to be run on each nodes 

 Don't forget to restart your node after running these commands !