Network hardware, especially high performance fabric is every bit as important as gpus and ram.

People seem to think Nvidia’s gpus are the be all and end all of enabling large models but there isn’t a large model out there that runs on a single gpu.

Broadcom sells things like pcie switches. They allow you to plug your in about a hundred gpus into one big switched pcie plane that can all communicate at full pcie bandwidth to each other. Nvidia has nvlink which is similar. You need this technology to enable large models and they sell for even higher margins than the gpus and ram.