> A bit like trying to compress a JPEG in a zip file
Interesting comparison, I wonder if there are loss-less compression schemes that are "weight-aware", about the statistical properties of model weights.
Back to that line - because (Huffman coded) JPEG images can be lossless recoded using arithmetic coding, which makes them smaller than if they were compressed with the best general-purpose compression algorithms/tools (e.g. `jpegtran -arithmetic < input.jpg > output.jpg` beats `xz -9 < input.jpg > output.jpg`). AC takes advantage of the properties of the image coding better than general-purpose compression tools on the output of the Huffman coding.
The library I use for archiving compressed raw LLM weights ([1] , posted in my other comment) afaik at least reorders the weights to more easily compress them. They also have a paper linked, which I admittedly didn't read, that might have more background and tradeoffs. Likely they will have discussed methods to take advantage of the structure of weights.
[1] https://github.com/zipnn/zipnn#results