alt Hacker News1.2TB/s is 2/3 the speed of an nVidia 5090.
But you get a generic computer and much more RAM.
And you lose a couple of organs.
Replies
It's worth noting that the 5090 (or the RTX Pro 6000 big brother with 92GB VRAM) will run rings around the Mac when it comes to compute.
My old 3090 is typically significantly faster (almost 2x token/s) than my M4 Max 128GB machine, as long as the model fits in the 24GB of VRAM.
In most situations it's a better idea to just buy tokens. But there are definitely cases when that's not an option. And then a machine like the M5 Ultra can allow you to do things locally for a fairly limited budget. And in a simpler package to manage than a machine with multiple GPUs.
is it still effectively 2/3rds? Don't know enough to compare a discrete GPU/CPU setup to something like this where it's more integrated
The real downside for me is not having Linux support.
It would take Apple one or two engineers to make Linux life much easier on macs. But Linux is outside their walled garden so it's ignored.