alt Hacker Newshow would you build memory into the weights? and why is RAG not enough? Our hippocampus is at a bit of a distance from our frontal cortex.
how would you build memory into the weights? and why is RAG not enough? Our hippocampus is at a bit of a distance from our frontal cortex.
Yeah it's a good question, I've also been thinking about harnesses and all these tacked on things we've done to add persistent memory, what makes that different, I don't know the answer, I guess that still 'feels' different than what we have, but it's hard to articulate how. As for the memory into weights thing, I meant along the lines of the Google TITANS/MIRAS papers that were released I think late last year.