We brute force AI models right now because a) we don’t know better, and b) it’s premature optimization.

I beg to differ on point b, but no one is delaying their next model just so they can concentrate on optimization.

It’s coming though.

One example: https://siliconangle.com/2026/07/28/ai-model-compression-sta...

Another is separating the the intelligence part of the model from the known facts part of the model (which can be better compressed)

Compression doesn't really work for model weights.

Model quantization and model distillation are two techniques to reduce model size.

one would hope model weights are already high entropy enough and would not be improved by a general-purpose memory compression algo?