Latency numbers every programmer should know

As @vitus points out [1], these numbers are pure, unadulterated garbage. Instead of being data, they are wildly incorrect extrapolations from this [2][3] actually useful and relatively correct past data.

2,000 bytes over commodity network in 5 us is 400 MB/s (3.2 Gbps). Far faster than commodity residential speeds.

1,000,000 bytes from memory in 741 ns (ridiculous amount of significant figures) is 1.3 TB/s. Modern DDR5 clocked at 4800 MHz is ~80 GB/s. Only a full stack of HBM3e (the memory they use on 10,000$ GPUs to saturate thousands of processing units) can achieve that.

1,000,000 bytes from SSD in 12.245 us (ridiculous amount of significant figures) is 81 GB/s. That is main memory speeds to a SSD. Actual NVMe SSDs are closer to 1-3 GB/s. Even PCIe 5.0 x16 has insufficient bandwidth to support that and most drives only support x2 or x4. For a x4 to support that, you would need a hypothetical PCIe 8.0 (expected to be finalized in 2028 with first implementations likely no earlier than 2030). And that is even ignoring the fact that the SSD itself is nowhere near being able to handle that level of bandwidth.

Disk seek in 1.649384 ms (ridiculous amount of significant figures) means we now have 36,000 RPM HDDs. Normal HDDs are 5400 or 7200 RPM resulting in 10 ms seek times.

Read 1,000,000 bytes from HDD in 358.968 us (ridiculous amount of significant figures) is 3 GB/s. That is NVMe SSD speeds on a SATA HDD. Actual HDDs are closer to 300 MB/s.

Every single one of these numbers is wildly incorrect with no basis in reality. This page is unequivocal anti-knowledge.

[1] https://news.ycombinator.com/item?id=47196505

[2] https://brenocon.com/dean_perf.html

[3] https://www.cs.cornell.edu/projects/ladis2009/talks/dean-key...

It’s just a copy of https://colin-scott.github.io/personal_website/research/inte... ?

There are many references, it's more important to have a general sense of what's happening and the relative costs than hard numbers (which tend to shift as hardware gets closer to ideal and software becomes worse at avoiding bloat).

https://github.com/donnemartin/system-design-primer?tab=read...

This graph is also very helpful

http://ithare.com/infographics-operation-costs-in-cpu-clock-...

I'd rather have them all as white squares, the colors break the sense of scale

> Send 2,000 bytes over commodity network: 5ns

Shouldn't this be 5µs?

Does anyone have Jeff Dean's sources for how he computed these numbers? What's the margin of error? How accurate are they now? Is there a set of numbers that also talks about memory bandwidth in GPUs? Are these numbers intel/amd only? How do they differ between an m1 architecture?

I don't think these numbers mean much to me but, I didn't know this site existed. What an excellent idea.

Can we also add: RDMA (RoCEv2) : ~2.5us