alt Hacker NewsWhat would be the non Mac computer to run these models locally at the same performance profile? Any similar linux ARM based computers that can reach the same level?
Replies
Framework Desktop is the closest one with the MAX 385/395 chip. It's mostly about the memory being fast enough rather than just CPU/GPU oomph.
The 64GB model is 2240€ base and the 128GB is 3069€ base + all the stuff you need to add to make it an actual computer.
As a comparison the 64GB Mac Mini is 2499€ here and a 128GB Mac Studio is 4274€.
Intel’s doing interesting things with their Arc GPUs. They’re offering GPUs that aren’t super fast for gaming but are relatively low power and have a boatload of VRAM. The new B70 is half the retail price of a 5090 (probably more like 1/3rd or 1/4 of actual 5090 selling prices) but has the same amount of memory and half the TDP. So for the same price as a 5090 you could get several and use them together.
I wonder if the Snapdragon X Elite already caught up with the Apple's M series in that regard - does anybody know?
Not even close. If you want to run this on PC's you need to get a GPU like 5090 but that's still not the same cost per token, and it will be less reliable and use a lot more power. Right now the Apple Silicon machines are the most cost effective per token and per watt.