How useful would that actually be for local inference, when the bottleneck is, essentially, always memory size? I think the sweet spot is somewhere around 64 to 128 GB. That ought to be enough for almost all uses, like Qwen3.8-Flash-Next at a decent quant.

Plus with more low precision compute… to help with the memory bandwidth!

More ASIC Mat-Mul too.