You could do lots of interesting things with sufficiently deep inter-layer integration.

For example, why not swap out not by LRU page but by dense node clusters on the heap graph, maintaining in-memory summaries of inbound and outbound edges for liveness? If you do this, you don't have to swap the cluster in to do a GC involving it.

If the whole cluster becomes unreachable, you wouldn't even have to swap it back in to get rid of it: you'd just drop the swap reference and deem the swap space free.

I don't see anything this deeply integrated happening near-term, but it's fun to think about.