logoalt Hacker News

DominikPeters • 01/21/2025 • 1 reply • view on HN

Indeed, for each of the words it got it right.


Replies

bt1a • 01/21/2025

How excellent for a quantized 27GB model (the Q6_K_L GGUF quantization type uses 8 bits per weight in the embedding and output layers since they're sensitize to quantization)