alt Hacker NewsOut of curiosity, what's makes the 125B unsuable? (performance of running it, the quality of that version of the model, or something else?)
Out of curiosity, what's makes the 125B unsuable? (performance of running it, the quality of that version of the model, or something else?)
No, it is actually very good. Qwen Flash 3.8 Next is fine. But you need ~128GB of RAM to get it going and not a lot of people have that or can serve it very quickly. I have been running it on an old gaming system around 25 t/s to do overnight work and it is very strong, even at 3 bit quant.