alt Hacker NewsThere's no inherent, technological reason to feed the historic data into the prompt, though. You could train a network with a second input for model-states, and during inference, feed into that input historic model-states that occured for the same LLM when consuming input that is somehow associatively related to whatever input is currently being consumed. Can you definitely show that this would be meaningfully different from how humans "experience" memories?
That would still be circumventing. There are other architectures that could be more dynamic, LLMs are not.
It is completely different. Our brain rewires itself to store memories, there is no memory storage for humans.