SQLite definitely seems like a poor choice for dealing with many concurrent requests.
Maybe it's improved since I last used it, but to my knowledge SQLite essentially forces all writes to be serialised, at risk of data corruption otherwise.
There are tricks for improving the performance such as WALs, but that is merely a performance boost rather than genuine concurrency with things like row-level locks that you might find in other databases.
I guess if the whole thing is architected with a write-through cache that handles concurrent writes and deals with serialising all the writes, then it can be a single writer streaming changes through to the database, but then you still have a point of serialisation, it just will manifest itself slightly differently.
And SQLite is something that will give you constraints you will always have to consider.
Whereas running mariadb or postgres on the same machine would deliver similar benefits without a risky migration.
If your DB is small enough to run as a SQLite database, then it probably ought to have never been on a different machine in the first place.
There is a very happy medium between SQLite and a database on a different machine, one I am continually surprised to see people ignore.
Can't you do infinity read replicas? I don't think the write load on lobster.rs is severe - maybe 1 comment/sec, with an acceptable replication delay of 60 sec?
If you can do that, you can have infinity instances.
Depends on the architecture..
They could have one SQLite instance per user and then have a single sweeper that goes through all last writes and then replicates it to the main instance - eventual consistency fanned out across files
> SQLite definitely seems like a poor choice for dealing with many concurrent requests
Can you qualify "many"? SQLite easily handles 100k+ writes per second and it's not hard to have app layer code serialize and batch writes to take advantage of that speed. Concurrent writes require a ton of overhead and your system and code can be quite a lot faster and simpler if you just skip the idea of them altogether.