alt Hacker NewsThe way I think of it has evolved a lot over the last 5 years. At this point I think human brains probably do something analogous to next token prediction when we think. For all the hype, I think LLMs are actually more, not less, intelligent than that average person realizes. I think it’s legit, actual intelligence, not just “artificial” intelligence. That may be a hot take but it’s just my perception.
Replies
We have spatial / quantitative and social / emotional aspects in our intelligence that are not at all like next token prediction.
If LLMs had shame, they'd surely not repeat mistakes (in the same context window) as much as they do.
It's language. Language itself is the thing that makes us smart in the unique way that we are among the other animals, and it weirdly turns out to be transferable to machines to at least some degree.
> I think it’s legit, actual intelligence, not just “artificial” intelligence. That may be a hot take but it’s just my perception.
You might be redefining words here; there isn't a form of intelligence that isn't actual intelligence. It is all actual intelligence. Artificial in this context means it is something we're creating in a lab. LLMs can't avoid being artificial intelligence. The meaning of "AI" is to artificially create actual intelligence.
average person is absolutely awful judge on anything you put in front of average person tho.
And if anything, average AI user is vastly overstating how good/useful it is. Papers about it pretty much always show huge gap between "productivity person thinks they are achieving" and "actual growth of productivity"
> At this point I think human brains probably do something analogous to next token prediction when we think
That's reasonable, but it doesn't mean that LLMs are close to being brains.
For a start, when humans think/talk, we often think ABOUT something - whatever is swirling about in our mind, or what we are currently seeing/feeling/etc. An LLM generating tokens/words is doing so only based on it's weights and the word sequence it is currently generating ... the human parallel would be more like a rapper spitting out words based on prior words, essentially on auto-pilot, or when we get triggered into spitting out stock phrases like "have a nice day".
If you want to compare an LLM to a human brain, it's basically equivalent to our language cortex if you ripped out all the external connections and ripped out all the feedback paths that make it capable of learning.
Of course there is a lot more to our brain than just our language cortex, but that alone should make you realize there is no real comparison beyond the fact that our language generation is also going to be based on prediction, and partly auto-regressive.