alt Hacker NewsAll of their benchmarks are against 16 bit models right?
Why aren't they comparing to 2/3/4 bit quants?
Replies
mstr_anderson • today at 6:02 AM
[flagged]
All of their benchmarks are against 16 bit models right?
Why aren't they comparing to 2/3/4 bit quants?
[flagged]
looked at quant versions of these models and they all outperform it so I guess it just doesn't look as good.