AMD unveils Threadripper Halo Station, an AI workstation packing 96 cores

The previous 'personal' AI Station I had pictured was the a16z one[0].

Each MI350P[1] in the TR Halo Station has 144GB VRAM and with 4.6 PFLOPs peak MXFP6 performance.

Four liquid cooled? Yes please.

[0] https://a16z.com/building-a16zs-personal-ai-workstation-with...

[1] https://www.amd.com/en/products/accelerators/instinct/mi350/...

This looks like a very cool high end x86 workstation. CPU alone is $12000 retail.

Theres a few potential “problems” I see with it - main system RAM is about 30-40x slower than that of the GPU’s and you will have PCI-e as a bottleneck getting working data to or between the GPUs.

In practice is the 400-500GB/s main memory limiting some use cases, or is the PCI-E bus the main issue with these types of systems?

So there's a path to 4 in order to run trillion parameter models? But you can't buy them or fit them in the case? Maybe in the Halo 2...

That all said, running frontier models locally offline would be very nice