alt Hacker NewsThe fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness.
It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember anything. It doesn't sit around thinking on its own.
Even if the model itself were extremely complex, it's hard to imagine a definition of consciousness that includes something that doesn't remember and can't change.
Replies
There are people whose brains don’t form new memories anymore after an accident or surgery, and they eternally live in the time before it happened, and have no memory of what happened a minute ago. Still they are conscious.
> It's not changed by the experience
The entire file is not changed, but the KV cache is.
> It doesn't remember anything
The model definitely remembers previous exchanges within the same conversation.
But you could argue the brain is just a bunch of coordinates describing spatial relationships between tokens too.
- average Hacker News response
Reinforcement learning changes the model. So it can and does change and remember based on experience. Eventually reinforcement learning can happen in real time.