alt Hacker News> The caveat is that Litestream replication is asynchronous. A restore can miss the newest local writes if the SQLite volume disappears before they are copied. That is fine for many AI and experimentation workflows
In short: SQLite is not all you need, unless you’re just experimenting don’t actually care about durability, in which case you also need litestream + object storage.
Right.
Replies
Postgres also does not synchronously replicate for free. You can setup both to get a confirmation write if you require that durability.
S3 is strongly consistent, if you need it anyways you can just use s3 keys to deconflict and store the workflow state.
"Durable workflows without the durability"
That's distributed workflows :)
i mean it's durable as long as nothing crashes or litestream has a data corruption bug which only happens every other release...
The suitability of Litestream for production disaster recovery is also an open question in my mind. I used 0.3.x for several years and when I tried to upgrade to the 0.5.x series there were runaway disk usage problems that would have caused downtime had they made it to prod. As far as I can tell these have not been entirely addressed, although recent bug reports suggest that they might be getting closer.
I want to love it, and I don't take open source projects like this for granted. But during my last production upgrade I chose to decommission Litestream in favor of a dumber, less granular solution using sqlite3_rsync and nightly backups because there is no point in using a backup system that is not rock solid.