alt Hacker NewsThat’s the problem - it seems like a mind but it doesn’t operate like the ones we’re used to.
Even a dog will learn from recent stimuli, these things don’t. The prompt just modifies.
Replies
operatingthetan • last Sunday at 11:14 PM
Yep, the only way these things can have "memory" is by shoving previous conversations into the context window.
➕ show 1 reply
whstl • last Sunday at 11:21 PM
I don't think that's a problem here at all.
The problem here is not doing tasks and outputting garbage output.
➕ show 1 reply
That's only because we hardcoded their weights in our implementation.
Aside from the cost, nothing about an LLM prevents feeding recent stimuli in and using it to update the models/retrain.
One can even do it in a makeshift way without modifying the weights, just keeping a complete version of any prompt + vector search on disk memory of it.