alt Hacker News<When WhatsApp pushed the Erlang BEAM virtual machine to its limits on 100+ core machines, the system choked. As detailed by Robin Morisset, idle threads trying to steal work spent all their CPU cycles fighting over the global runq_lock5>
A simple burst of memmap + soft fault with 100 or 1000 threads on a normal laptop would tell you that thread contention is real and cache locality gets destroyed. Couple that with pinned threads. You can see the latency increase by increasing thread count. Add to that the motherboard interconnect tax for numa systems. Work stealing is not the way for increasingly many workloads on modern hardware.
Recently we built Dip, our in-house ephemeral + parallel database, and we went with may coroutines + work pinning to the same thread which also nicely becomes numa aware via architecture.
Increasing threads beyond system's hardware cores/threads resulted only in marginal gains of a couple of milliseconds worth of differences on huge workload with large increase in memory (thread stack) used by the massive number of threads.
Replies
> Increasing threads beyond system's hardware cores/threads resulted only in marginal gains of a couple of milliseconds worth of differences on huge workload with large increase in memory
Careful, if you say that too loudly, the "get rid of async just spawn more threads!!!!!" people will come out of the woodwork to yell at you about how _all_ async is a lie and we should instead pretend none of it exists and just spawn more threads.
The Erlang thing is different though - that's a userspace scheduler, scheduling userspace tasks onto one-thread-per-core. Classic M:N scheduling in a similar way to go.
It works extremely well at scale - you can handle 10k connections on a normal machine with very little thought, and WhatsApp has reported handling 1 million. Yes everything has some point at which it won't scale, and apparently that approach (or at least that implementation) struggles at 100 cores. That's not a normal machine.