alt Hacker NewsThe problem with that is LLMs can output words or symbols that seen like it used "reason" to produce. But for everything the core algorithm does, it's simply nothing like the wetware reasoning to get to the same answer. So he didn't move goalposts. He always meant the reasoning that stems from human cognition.
Technically if it has that, it'd be singularity no? So basically the premise is they are doing nothing of the sort. Prove any LLM enough and it really does show it has no quarrels contradicting itself or being bossed around. Has no belief / no orientation etc. It's truly mindless but tricks our mind and soul (or whatever) probably.
> Technically if it has that, it'd be singularity no?
reasoning is not black and white. It is possible to reason poorly. Most people cannot do basic math proofs, even math majors struggle with the hardest math proofs. Reasoning in humans is also context/token dependent. I just spent one HOUR trying to show my mom (who has mild dementia) how to use amazon fire (push DOWN until your channel shows up, push RIGHT until the channel becomes big) and she could not figure it out. Rewrote the instructions in japanese and she followed the logic relatively smoothly. Ironically, i'm pretty sure her english is better than her japanese, vocabulary wise.
> it's simply nothing like the wetware reasoning to get to the same answer.
but you don't know how wetware reasoning works, so you are incapable of making that proclamation. I'm pretty sure when I do math proofs (I'm not an amazing mathematician) sometimes I have to literally tick my way through each step of the proof, sometimes breaking it down to super-basic substeps, which to me feels awful lot like what an LLM could be doing. For that matter we don't know how LLM reasoning works but my claim is that these LLMs are in principle capable of reasoning due to architecture.
If this doesn't make sense I suggest you look over the architecture of LLMs carefully and try to understand my point.
(BTW I'm not talking about "reasoning models" with "thinking turns", that's just marketing speak, I'm talking about ANY transformer-based model, even the "dumbest UX architecture" completion models)