alt Hacker NewsSo the argument of this _very_ Claude written piece is that.. On apple devices you have GPUs with fast-enough unified memory and can link them together?
Okay, fair, but why would I get an Apple device when I can get a Linux device with some monster motherboard that has a dozen memory slots?
Replies
smw • today at 5:12 PM
Because bandwidth between those sticks and the GPU isn't fast enough for fast inference.
➕ show 1 reply
This made me go down a rabbit hole.
Looks like Mac Minis had a 64 GB model (hard to get now) for roughly $1,600. You could get a 3 of these (192GB) & thanks to the Thunderbolt 5 connection, they out performed any single board AMD for a similar price that had more memory. If you tried to do so something similar with Nvidia GPU you're looking at a lot of watts, cooling & a much faster experience but only 48 GB of memory.