You think they introduce stronger quantization after a few days?
For sure they quickly move to q8, the output quality difference to bf16 is small compared to the speed/capacity gain
alt Hacker News
For sure they quickly move to q8, the output quality difference to bf16 is small compared to the speed/capacity gain