but why would you ever want to run ceph? its just such a huge monster.

Its also not that useful even if you have enough machines to run it properly.

NVME and zfs is fast enough for virtually anything now. With snapshot and snapshot sending you get decent backups for half the hardware cost of ceph.

Has anyone that has set up microceph determined the overhead of the required multiple OSDs? The docs make it sound scary, but it's not clear if that's because people run it on a Pi with an sdcard for block storage or because someone once ran 18TB of OSDs in production that then fell over.