alt Hacker News> suspect it’s better than say a 9b or 12b model
Whaaat, a 27b model might be better than 9b or 12b model? What would make you do such an outrageous claim?
> suspect it’s better than say a 9b or 12b model
Whaaat, a 27b model might be better than 9b or 12b model? What would make you do such an outrageous claim?
Sorry I should have clarified - I meant that a ternary 27b model would outperform a non-quantized or 8 bit quantized 9 or 12b model - which it is generally close to (or much smaller than) in size. So yeah the comparison I was trying to make was between models of equivalent size or models that could run on similarly sized hardware.