alt Hacker Newswill be great fun if one M5 Ultra with 512GB memory at 1.2T bandwidth capable of doing 3x smallish local model inferencing each at Opus 4.5 level of intelligence.
will be great fun if one M5 Ultra with 512GB memory at 1.2T bandwidth capable of doing 3x smallish local model inferencing each at Opus 4.5 level of intelligence.
The memory bandwidth and size seems to be there, but what is the tokens per sec on like a qwen model? And you can basically do 3x opus 4.5 on the $100 a month claude plan. Your payback will be near infinity years after electricity.