alt Hacker News> Please do give that a try and report back the prefill and decode speed.
M4 Max here w/ 128GB RAM. Can confirm this is the bottleneck.
I weighed about a DGX Spark but thought the M4 would be competitive with equal RAM. Not so much.
> Please do give that a try and report back the prefill and decode speed.
M4 Max here w/ 128GB RAM. Can confirm this is the bottleneck.
I weighed about a DGX Spark but thought the M4 would be competitive with equal RAM. Not so much.
I think the DGX Spark will likely underperform the M4 from what I've read.
However it will be better for training / fine tuning, etc. type workflows.