> they would have to not lose context (or at least keep some).

I guess you can start to see that happen (for a while now) with e.g., ChatGPT, Claude, etc. keeping persistent memory of you, automatically.

Personally, I often find these memorization features annoying, because - more often than not - the persisted context is outdated because something in my real life changed that ChatGPT & co are not aware of.