alt Hacker NewsCould you go into more detail about this issue and provide some links to the documentation?
I'm absolutely no expert, I'm just reading about it now, but from the SQLite WAL documentation [0]:
> 7. There is the extra operation of checkpointing which, though automatic by default, is still something that application developers need to be mindful of.
and from their "checkpointing" documentation [1]:
> By default, SQLite does a checkpoint automatically when the WAL file reaches a threshold size of 1000 pages.
I'm only skimming but I see no mention of waiting for an idle time window.
Have you been successful in DoS'ing lobste.rs?
On the page you linked:
>However, if a database has many concurrent overlapping readers and there is always at least one active reader, then no checkpoints will be able to complete and hence the WAL file will grow without bound.
>This scenario can be avoided by ensuring that there are "reader gaps": times when no processes are reading from the database and that checkpoints are attempted during those times.
Dunno, maybe Rails has a built-in workaround for this.
My workaround was to run a separate thread that monitored the WAL size on disk every second. If it went above the target size of 8 MB, my framework would enter "slow down" mode, where all reads and writes were artificially delayed by calling "sleep()", starting at 16 ms and gradually increasing the sleep time based on a few heuristics.
This allowed the application to have short gaps with no reads or writes, so the checkpointer could actually proceed.