alt Hacker NewsI think understanding how this stuff works is really important. For technical people it gives them a useful starting point for understanding it all. For less technical people it's crucial to help them understand that it's not some weird new magical science-fiction AI - it's still computer programs that turn text into numbers and do stuff with the numbers and turn those back into text.
It's harder to believe something is conscious or threatening to achieve word domination once you understand that it's a machine that statistically figures out which word should come next.
Replies
While you’re active in this thread, I just want to say thank you for all your writing, you’re such a reliable source of sanity in that crazy new world :)
It's harder to believe something is conscious or threatening to achieve word domination once you understand that it's a machine that statistically figures out which word should come next.
The problem is, these models challenge our definition of "consciousness." Or at least they point out how hopelessly-inadequate our thinking on the subject is. Some people really, really don't like having their personal definition of consciousness challenged.
The correct response to "So what, it's just a next-token predictor" isn't a long dissertation on RLHF, training architectures, scaling laws and whatever, but rather to turn around and respond, "Sure, and how is that different from what we do?"
> It's harder to believe something is conscious or threatening to achieve word domination once you understand that it's a machine that statistically figures out which word should come next.
At the risk of sounding overly flippant, all world domination has been achieved by some person(s) figuring out which word should come next. Words quite literally = action when it comes to LLM’s with tools access
I don't think the next-token-predictor thing should increase anyone's confidence that LLMs aren't conscious or can't escape the control of their operators. A very closely analogous argument would "prove" that humans aren't conscious or can't do [insert task here] either. (No, I'm not saying that any of this is true of today's LLMs, I'm saying this particular argument doesn't work.)
I recommend this explanation: https://www.astralcodexten.com/p/next-token-predictor-is-an-...