logoalt Hacker News

throw-qqqqq • today at 3:55 PM • 0 replies • view on HN

The max theoretical ratio of (byte-independent) compression can be approximated using the Shannon entropy.

Most quantized GGUFs have a very high Shannon entropy, so are almost perfectly compressed already. A Q4_K_M quantization is often around +7.90 bits/byte and so is virtually incompressible.

Unquantized models (F16) are often around +7bits/byte for comparison.

https://en.wikipedia.org/wiki/Shannon%27s_source_coding_theo...