12 channel ddr5 5600 ECC is around 500gbs which in real world works very well for large MoE
You mean 500 GB/s, not Gb/s (actually 537 GB/s).
Unfortunately that does not matter. Even in a cheap desktop motherboard the memory bandwidth is higher than of 16-lane PCIe 5.0.
Therefore the memory bandwidth available to a discrete GPU is determined by its PCIe slot, not by the system memory.
If you install multiple GPUs, in many MBs that will halve the bandwidth of the PCIe slots, for an even lower memory throughput.
alt Hacker News
You mean 500 GB/s, not Gb/s (actually 537 GB/s).
Unfortunately that does not matter. Even in a cheap desktop motherboard the memory bandwidth is higher than of 16-lane PCIe 5.0.
Therefore the memory bandwidth available to a discrete GPU is determined by its PCIe slot, not by the system memory.
If you install multiple GPUs, in many MBs that will halve the bandwidth of the PCIe slots, for an even lower memory throughput.