> You will need at least 75 GB of RAM or unified memory to run the model. Its smallest 1-bit quantized version is larger than usual because of the model’s architecture so 1-bit isn't really 1-bit at all. However, this also means the quantization is less aggressive, allowing the model to retain more of its original accuracy than more heavily quantized models.
Lots of RAM required even for the 1-bit, which is already downloadable. Interested to see how well this one works compared to Ornith1.5-35B-A3B I've been running (and quite happy about).
alt Hacker News
They are already working on it.
https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF https://unsloth.ai/docs/models/qwen3.8-next
> You will need at least 75 GB of RAM or unified memory to run the model. Its smallest 1-bit quantized version is larger than usual because of the model’s architecture so 1-bit isn't really 1-bit at all. However, this also means the quantization is less aggressive, allowing the model to retain more of its original accuracy than more heavily quantized models.
Lots of RAM required even for the 1-bit, which is already downloadable. Interested to see how well this one works compared to Ornith1.5-35B-A3B I've been running (and quite happy about).
Edit: but llama-cpp does not yet support it.