alt Hacker NewsI don't think the next-token-predictor thing should increase anyone's confidence that LLMs aren't conscious or can't escape the control of their operators. A very closely analogous argument would "prove" that humans aren't conscious or can't do [insert task here] either. (No, I'm not saying that any of this is true of today's LLMs, I'm saying this particular argument doesn't work.)
I recommend this explanation: https://www.astralcodexten.com/p/next-token-predictor-is-an-...
You can say that for any argument regarding consciousness, because we don’t have an actual, all encompassing definition of what consciousness is. In general I don’t think comparison with humans makes much sense, we should be able to discuss LLMs without always falling back to “but what about humans” (sorry for the caricature)