Btrfs/ZFS/bcachefs under workloads classic benchmarks skip

The author of the benchmark here. I went over some comments and I'll try to tackle them here. I'm pretty clear that GH runner based benchmark is far from perfect due to noisy neighbours etc. Thus every test first is running so called calibration... to reject completely unreliable VMs. I'm fully aware that this can't completely fix the issue. Can limit it but not fix. But as of now there are 593 runs recorded so average should still be quite meaningful.

Having that said I'm desperately trying to get REAL hardware to run that benchmark. With some successes ;)

Few months ago I got Hetzner machine from Kent Overstreet and I was able to finish 3 runs before machine died... Results: https://bartosz.fenski.pl/modern-fs-benchmark/real-hw/

Currently I've got even more interesting machine with tons of disks and I'm running new set of benchmarks but it's really in its initial stage.

https://bartosz.fenski.pl/modern-fs-benchmark/sas-hdd/ 2nd run in progress... one run on REAL hardware takes much more time than on GH runner so it's slow.

But this new hardware has also so many disks that the plan is to try also more complex, tiered cache topologies. I'm working on it.

I'm happy to answer any other questions, sources of every piece of this benchmark are freely available and I'm not saying they are 100% correct. I'm open to improvements.

> CI runs use loop devices on shared ephemeral VMs (one VM per filesystem): compare shapes and ratios, not absolute MB/s. Each job records a host-calibration anchor — see the table.

I think if you're not using baremetal for such tests, it's likely that the results are simply not comparable at all? What if another tenant is also using the disk?

> Every push/2-hourly cron builds each filesystem across 4 loop devices backed by sparse files, runs the suite, and publishes a results table in the job summary plus JSON artifacts.

I get that real hardware costs (author mentions EUR 70 a month for a suitable server), but without at least a baseline snapshot comparison run between real hardware, both SSD and HDD, and the sparse file-backed loop devices, it's hard to take much away from this.

Sadly the AI apocalypse isn't making stuff like this easy to do as a hobby.

What is md-raid10 doing that is so much worse than lvm-raid10? In terms of "I/O" and "responsiveness." It's not really obvious to me from either the linked page or https://github.com/fenio/modern-fs-benchmark . In principle they should be similar?

Oh, so bcachefs is doing pretty well.

So two filesystems that are essentially shunned from the Linux kernel and permanent second-class citizens, and one that was removed from Red Hat and has a questionable history of reliability. Oh boy which do I choose?

I'm saying ZFS on another OS.

https://github.com/fenio/modern-fs-benchmark

For peace of mind I'm still using zfs (since the last 15+ years) but I'm definitely not impressed by the performance...

I think the reviews should also include the social aspect of these filesystems...

There is and have been many promising and exciting FS to replace the old boring ones, but for storage you not only want to avoid technical issues but also maintainer(s) drama...

what does "integrity" fail mean in the first table? that the case didn't recover from the 2G corruption?

@farlight assuming that you're the creator do you think you'd be able to rework the HTML/CSS? I'm sure you've got good data but speaking on behalf of my eyeballs, the results page is... hard to read!