alt Hacker NewsFor $50K, you could buy 25 Framework desktop motherboards (128G VRAM each w/Strix Halo, so over 3TB total) Not sure how you'll cluster all of them but it might be fun to try. ;)
Replies
3abiton • last Friday at 11:42 PM
You could use llama.cpp rpc mode over "network" via usb4/thunderbolt connection
There is no way to achieve a high throughput low latency connection between 25 Strix Halo systems. After accounting for storage and network, there are barely any PCIe lanes left to link two of them together.
You might be able to use USB4 but unsure how the latency is for that.