Making memory is easy. Packaging that memory within a few millimeters of a piece of silicon using TSVs and maintaining signal integrity on a 1024 bit bus is really, really hard.

LLMs aren’t all that compute constrained or even memory constrained. It’s just that pushing dozens of terabits per second through a piece of silicon is a physics problem.