alt Hacker NewsThis is completely wrong. First, disabling overcommit is wasteful because of fork and because of the way thread stacks are allocated. Sorry, you don't get exact memory accounting with C, not even Windows will do exact accounting of thread stacks.
Secondly, memory is a global resource so you don't get local failures when it's exhausted, whoever allocates first after memory has been exhausted will get an error they might be the application responsible for the exhaustion or they might not be. They might crash on the error or they might "handle it", keep going and render the system completely unusable.
No, exact accounting is not a solution. Ulimits and configuring the OOM killer are solutions.
Replies
> because of fork and because of the way thread stacks are allocated
For modern (post-x86_64) memory allocators a common strategy is to allocate hundreds of gigabytes of virtual memory and let the kernel handle deal with actually swapping in physical memory pages upon use.
This way you can partition the virtual memory space into arenas as you like. This works really well.
Without going into a discussion about whether this is right or wrong, how is fork(2) wasteful?
fork(2) has been copy-on-write for decades and does not copy the entire process address space. Thread stacks are a non-sequitur either as stack uses the data pages, the thread stacks are rather small in size in most scanarios, hence the thread stacks are also subject to copy-on-write.
The only overhead that the use of fork(2) incurs is an extra copy of process' memory descriptior pages, which is a drop in the ocean for modern systems for large amounts of RAM.