alt Hacker NewsIt's annoying that most file formats don't checksum their own content.
Even formats which should know better, like SQLite, delegate that to the filesystem, most of which are also not checksumed and which delegate that further to the storage.
PostgreSQL, which prides itself by it's quality and reliability, only turned on checksums by default in the last version, 18.
This is one great benefit of using .zip files as file formats, you get this for free.
Replies
Checksums use CPU cycles. SQLite will do checksums with an extension (https://sqlite.org/cksumvfs.html) but that is off by default since an overwhelming majority of developers are more interested in day-to-day performance than detecting (very rare) storage malfunctions.
> It's annoying that most file formats don't checksum their own content.
I agree.
> Even formats which should know better, like SQLite, delegate that to the filesystem, most of which are also not checksumed and which delegate that further to the storage.
I mostly run ext4 (desktop, laptops, etc.) but for my main server at home, it's a ZFS (mirrored) tank on an old server with ECC RAM.
And backups. So much backups.
I think it's reasonable for a DB like SQLite to delegate that to the filesystem. There is an overhead for doing it on the DB level, and since SQLite is just a file on the filesystem which, presumably, is serving many other files as well, why would you trust anything else on the filesystem if you don't trust SQLite? Like, your PHP script (or nginx server executable, or whatever) that is calling SQLite, that's not going to be check-summed either. Either you trust your filesystem or you don't, and if you don't, checksum and error correct on the filesystem level.
Though fair enough, it could offer it as an opt-in thing.