Since you need to apply CPU time to compress memory, and also CPU before you can map pages back into process memory (by uncompressing), this is only useful for not frequently used memory, like swap only a bit faster.
I'm not too familiar with LLM memory access patterns, but since the model files are stored on disk, you wouldn't ever keep them in memory anyway (except caching) and just reread them on demand.
> I'm not too familiar with LLM memory access patterns
Dense architectures hit every byte of the model for every token. It's the ultimate memory bandwidth nightmare.
MoE models use subsections for each token and are more amenable to keeping parts in secondary storage, and indeed there are some local llm engines that do this.