alt Hacker NewsPersonal take: LLMs are probably part of the answer (to AGI?) but are hugely handicapped by their current architecture: the only time that long-term memories are formed is during training, and everything after that (once they're being interacted with) sits only in their context window, which is the equivalent of fungible, fallible, lossy short-term memory. [0] I suspect that many things they currently struggle with can be traced back to this.
Overcome this fundamental limitation and we'll have created introspection and self-learning. However, it's hard to predict whether this will allow them to make novel, intuitive leaps of discovery?
[0] It's an imperfect analogy, but we're expecting perfection from creations which are similarly handicapped as Leonard Shelby in the film Memento.
Replies
I've spent a few weeks building and using a terminal LLM client based on that RLM paper that was floating around a little while ago. It's single-conversation, with a tiny, sliding context window, and then a tool that basically fuzzy searches across our full interaction history. It's memory is 'better' than mine - but anything that is essentially RAG inherently will be.
My learning so far, to your point on memory being a limiting factor, is that the system is able to build on ideas over time. I'm not sure you'd classify that as 'self-learning', and I haven't really pushed it in the direction of 'introspection' at all.
Memory itself (in this form) does not seem to be a silver bullet, though, by any means. However, as I add more 'tools', or 'agents', its ability to make 'leaps of discovery' does improve.
For example, I've been (very cautiously) allowing cron jobs to review a day's conversation, then spawn headless Claude Code instances to explore ideas or produce research on topics that I've been thinking about in the chat history.
That's not much different from the 'regular tasks' that Perplexity (and I think OpenAI) offer, but it definitely feels more like a singular entity. It's absolutely limited by how smart the conversation history is, at this time, though.
The Memento analogy you used does feel quite apt - there is a distinct sense of personhood available to something with memory that is inherently unavailable to a fresh context window.
MIT have developed a technique called Self-Adapting Language Models (SEAL), which enables LLMs to continuously improve by generating their own synthetic training data and updating their internal parameters in response to new information.
ToolAlpaca, InterCode and Reflexion are taking different approaches among others.
LLMs of tomorrow will be quite different.
FWIW there's already a number of proposals for augmenting LLMs with long-term memory. And many of them show promising results.
So, perhaps, what's needed is not a discovery, but a way to identify optimal method.
Note that it's hard to come up with a long-term memory test which would be different from either a long-context test (i.e. LLM remembers something over a long distance) or RAG-like test.
It’s also hugely handicapped because it cannot churn in a continuous loop yet. For example, we humans are essentially a constant video stream of inputs from eyes to brain. This churns our brain, the running loop is our aliveness (not consciousness). At the moment, we get these LLMs to churn (chain of thought or reasoning loops) in a very limited fashion due to compute limitations.
If we get a little creative, and allow the LLM to self-inject concepts within this loop (as Anthropic explained here https://www.anthropic.com/research/introspection), then we’re taking about something that is seemingly active and adapting.
We’re not there yet, but we will be.
I'm also reminded of the bit from Neuromancer where Case removes and then reinserts the Dixie Flatline "ROM construct" cartridge, resetting Dixie to the moment just before his death and causing him to forget their previous (albeit brief) conversation. Dixie can't meaningfully grow as a person. All that he ever will be is burned onto that cart; anything he learns since then is stored in temporary memory. Perhaps this is part of the reason why he wishes to be erased forever, ending his suffering.
Yes, but it's not just memory hierarchy on which plain transformer-based LLMs are handicapped, there are many deficiencies. (For example, why must they do all their thinking upfront in thinking blocks rather than at any point when they become uncertain?) I'm not sure why you link memory to introspection.
This is why so many people (especially those that think they understand LLM limitations) massively underestimate the future progress of LLMs: people everywhere can see architectural problems and are working on fixing them. These aren't fundamental limitations of large DNN language models in general. Architecture can be adjusted. Turns out you can even put recurrence back in (SSMs) without worse scalability.