alt Hacker NewsI think what makes FUSE/FSKit great here is that agents usually only need to see the file metadata + read a handful of actual files, rather than some applications needing read many things. If you're doing huge rewrites, this is a problem, but most tasks are usually somewhat small. Definitely is a problem that I've ran into though, we do cache aggressively to try to solve some of this, but it'll never be as fast as reading/writing directly to disk. We have benchmarks [1] if you want to take a look at how we're testing some of the performance there.
Couldn't run your benchmarks, as I did not create an account, and a bit of a different beast that I am comparing against (P2P distributed filesystem), but these are my numbers and setup, and the on-demand part lines up with what I have observed:
Setup: a Linux box on the other side of Romania (compared to where I am living) reading from a Windows box in Singapore (~200 ms RTT)
- reading 1 MiB of a 1 GB remote file pulls only 16 MiB (~98% avoided) - this is because of my fine tuning optimization choice - first byte approx: 2.3s - git-LFS repos also clone cold over the mount byte-perfect (separate Mac - Linux run on a ~20 ms RTT)
The thing that I do differently is that my metadata is eagerly pushed, as I optimized for content streaming.
And 100k-file tree mounts I did not test yet.
But my goal was to have instant file access for generic files between apps, and peer to peer, supports also Windows :D
here is the tool: https://github.com/KeibiSoft/KeibiDrop