I'm running a max of 1M context locally. I think the next real move will be continuous learning/infinite context. That will be another game changer.
Infinite context requires Infinite memory.
Infinite context requires Infinite memory.