I don't know at all. But I've been playing with oMlx and Splash. Using Pi there are compression extensions like the billion-context. I suspect your right.
But all seem to have massive issue boards. What I did see was oMlx release clustering so you can string 3 macs and have 3 times the memory or use one for prefilling accelerating.
Which I thought was neat and what I'll likely poke at instead of getting more ran.
perhaps i'm not pilled enough for this conversation, but aren't you conflating two disparate concepts — model-weights and context-window?