alt Hacker NewsAt the most basic level - LLMs are stateless machines. They have no shared world view other than the weights encoded in the model (the knowledge “cut off”)
Anything else must be fed as context- therefore, if you feed an LLM a fresh query with no context, there is no danger that it would have access to context from another session.
Basic web application session management applies here. Doesn’t mean that trillion dollar valued companies can’t mess it up tho. https://www.bitdefender.com/en-us/blog/hotforsecurity/chatgp...
Yeah despite the conceptual statelessness, there is quite a bit of state that hangs around though--KV cache and context. I still haven't been able to find anything concrete in docs about how these are isolated. In any case it's clearly a different class of issue than the one from the article. Not endemic to how LLMs work, just normal web session stuff, modulo some GPU memory handling.