I haven't tried it in RAM, but your reasoning matches what I saw when I looked at compressing weights for archiving them. The weights don't compress nicely with standard, especially fast, algorithms. Some more background in this repo [1] including example compression ratios. I would also worry about decompression speed, that gives you a hard bound on the throughput. If the numbers in this repo are accurate, you likely hit the throughput limit of decompression way before you hit the memory limit, even considering the space savings and thus higher effective throughput.