alt Hacker NewsI think the future should be efficient quantization which can make model with the same level as Qwen 3.8 27b now accessible from our laptop.
I think the future should be efficient quantization which can make model with the same level as Qwen 3.8 27b now accessible from our laptop.
This is mostly a memory bandwidth issue. Qwen 3.8 27b and the related Bonsai 2 27B run on my M5 Max laptop with 36GB of ram. It’s a little slow due to my specific model having lower memory bandwidth than higher end models with more ram and cores. MoE models perform better here and expert streaming engines can allow larger than memory models to run.