alt Hacker NewsSometimes after a night’s sleep, we wake up with an insight on a topic or a solution to a problem we encountered the day before. Did we “think” in our sleep to come up with the insight or solution? For all we know, it’s an unconscious process. Would we call it “thinking”?
The term “thinking” is rather ill-defined, too bound to how we perceive our own wakeful thinking.
When conversing with LLMs, I never get the feeling that they have a solid grasp on the conversation. When you dig into topics, there is always a little too much vagueness, a slight but clear lack of coherence, continuity and awareness, a prevalence of cookie-cutter verbiage. It feels like a mind that isn’t fully “there” — and maybe not at all.
I would agree that LLMs reason (well, the reasoning models). But “thinking”? I don’t know. There is something missing.
Replies
There is simply put no ongoing process and no feedback loop. The model does not learn. The cognition ends when the inference cycle ends. It's not thinking, it just produces output that looks similar to the output of thinking. But the process by which it does that is wholly unreleated.
Interesting, you think the associations your brain comes up with during sleep are NOT thinking?
Do LLMs ever ask for you to clarify something you said in a way a person who doesn't quite understand what you said will do?
> When conversing with LLMs, I never get the feeling that they have a solid grasp on the conversation. When you dig into topics, there is always a little too much vagueness, a slight but clear lack of coherence, continuity and awareness, a prevalence of cookie-cutter verbiage. It feels like a mind that isn’t fully “there” — and maybe not at all.
Much like speaking to a less experienced colleague, no?
They say things that contain the right ideas, but arrange it unconvincingly. Still useful to have though.
Perhaps this is an artefact of instantiation - when you talk with an LLM, the responding instance is just that - it comes into being, inhales your entire chat history, and then continues like the last chap, finishes its response, and dies.
The continuity is currently an illusion.
> Would we call it “thinking”?
Yes I would.
> Sometimes after a night’s sleep, we wake up with an insight on a topic or a solution to a problem we encountered the day before.
The current crop of models do not "sleep" in any way. The associated limitations on long term task adaptation are obvious barriers to their general utility.
> When conversing with LLMs, I never get the feeling that they have a solid grasp on the conversation. When you dig into topics, there is always a little too much vagueness, a slight but clear lack of coherence, continuity and awareness, a prevalence of cookie-cutter verbiage. It feels like a mind that isn’t fully “there” — and maybe not at all.
One of the key functions of REM sleep seems to be the ability to generalize concepts and make connections between "distant" ideas in latent space [1].
I would argue that the current crop of LLMs are overfit on recall ability, particularly on their training corpus. The inherent trade-off is that they are underfit on "conceptual" intelligence. The ability to make connections between these ideas.
As a result, you often get "thinking shaped objects", to paraphrase Janelle Shane [2]. It does feel like the primordial ooze of intelligence, but it is clear we still have several transformer-shaped breakthroughs before actual (human comparable) intelligence.
1. https://en.wikipedia.org/wiki/Why_We_Sleep 2. https://www.aiweirdness.com/