Just an idea: perhaps all of the end devices should have at least some high reliability storage. This would enable local applications that require high data durability and integrity.
Probably it'd require ECC RAM to prevent in memory bitrot, multiple copies of blocks (or even multiple physical block devices) with strong checksums.
Perhaps this data should somehow "automagically" sync between all locally available devices, again protected with strong checksums at every step.
(This idea requires some refining.)
I wonder if you could combine it with BitTorrent to get the distributed nature
We've thought a fair amount about this. Our approach is the use sqlite on-device. Think about it more as a partially replicated db instead of a cache.
Then locally available devices can compare changelogs and sync only the delta.
No need for a checksum, since you can use monetonically increasing version numbers and CRDTs!