logoalt Hacker News

sreantoday at 1:16 PM1 replyview on HN

The more fundamental reason is Amdahl's law

https://en.wikipedia.org/wiki/Amdahl%27s_law

Even a tiny bit of serial instruction will limit the speed up


Replies

w-mtoday at 3:02 PM

The plot doesn't appear to be in Amdahl territory yet. The single-threaded time in the plot looks to be around 39 seconds. A perfect division into 32 workers without overhead would make it 39 / 32 = 1.22 seconds. With the multi-threaded workload being reported as 1.5 seconds in the text, there's still only .3 seconds of overhead + serial instructions that can't be parallelized.

Every doubling of the number of workers halves the execution time cleanly in the plot, from 40 seconds to 20 seconds to 10 seconds. Eyeballing this for 32 over 16 workers is difficult, but it still seems close to halving the total time once again. So there's not a lot of Amdahl flattening, it's just the plain physics of looking at a inverse-proportional curve.