> Wouldn't that be the commercial LLMs? ChatGPT, Claude etc are estimated to be in the 2-10 TB range, and a large portion of the population of developed countries are using those apps commonly. Not on their own systems, but if we're in the 1995 academic perspective, multiuser systems are assumed.

I think each part is generally treated like an individual large program - which makes sense because otherwise you have large parts of the model sitting there doing nothing. You want all of your silicon running hot, RAM sitting doing nothing for a period of time is a waste.

> If OpenAI had needed a single addressing plane to cover all their users, 128-bit might be required. Similar to how the internet hobbles along fine with 32-bit addressing by just adding a layer of indirection to the internet with NAT.

I think even then, 128 bit memory for OpenAI et al would be more of a headache than it is worth. Economically it's not worth building the 128 bit systems. The fact that servers and consumer CPUs have a lot of re-use within their designs reduces the costs for everybody. Sony for example stopped producing their 128 bit CPU [1], where it was mostly about pushing more data between CPUs and VPUs. Now we have specifications like PCIe where you can just adjust your pipe width to increase throughput.

> I guess the other example of 128-bit addressing is ZFS. What are the biggest ZFS file systems? And who has more data? S3? Once you get bigger than 64-bit you want to scale out horizontally anyway and not just put it all into one flat addressable plane.

Yeah exactly. The other thing to consider is that it doesn't matter how many bits wide you can go, the bottleneck is the smallest width. With ZFS, at some point you likely need to send it over a network, and I think fibre is up to 64 bits via parallel transceivers. If that points outwards, it'll probably be single bit by the time it gets to you.

[1] https://en.wikipedia.org/wiki/Emotion_Engine