alt Hacker NewsI’ve actually been really impressed with the 27b model they recently released - amazing performance approaching 40 tok/s on m4 max and I didn’t run into any quality issues in the small set of tasks I tried. Haven’t gone full coding with it yet but suspect it’s better than say a 9b or 12b model.
Replies
embedding-shape • yesterday at 10:51 AM
> suspect it’s better than say a 9b or 12b model
Whaaat, a 27b model might be better than 9b or 12b model? What would make you do such an outrageous claim?
➕ show 1 reply
I ran into some issues that are more extreme versions of Qwen's thinking loops while reasoning. It is great at the small puzzles I set for it but it did some frankly insane things on a PHP coding task I set it.
It also had some issues that might be parsing/chat template stuff, tool calling oddities. I will try it again, I did try it pretty much the day it shipped and it's possible there are more improvements in their llama.cpp fork since.
It would be churlish to be overcritical, mind you — the PrismML ternary stuff is an advance — but it feels like it should be applied at training. I figure we will see that, somewhere, quite soon.
Did you try the BottleCap ThinkingCap Qwen post-train with the reduced thinking overhead?