alt Hacker Newswith a huge multi-tenant database, how do you deal with noisy neighbors? indexes are surely necessary, which impose a non-trivial cost at scale.
with a huge multi-tenant database, how do you deal with noisy neighbors? indexes are surely necessary, which impose a non-trivial cost at scale.
One data structure that helps a lot is the grouped queue.
I cover it in the essay here:
https://www.instantdb.com/essays/architecture#:~:text=is%20t...
To summarize:
In places where we process throughput, we generally stick a grouped queue and a threadpool that takes from it. The mechanics for this queue make it so that if there's one noisy neighbor, it can't hog all the threads.
There's more too (runbooks, rate limiting systems, buffers, isolated instances), but I thought this particular data structure was really fun to share.