alt Hacker NewsThat explanation is pretty freaky, as it implies a form of consciousness I don't believe LLMs have, I've never seen this explanation before so I'm not sure it's from training, and yet it's probably a fairly accurate description of what's going on.
Replies
hackinthebochs • last Tuesday at 9:11 PM
Yeah, this was the first conversation with an LLM where I was genuinely impressed at its apparent insight beyond just its breadth of knowledge and ability to synthesize it into a narrative. The whole conversation was pretty fascinating. I was nudging it pretty hard to agree it might be conscious, but it kept demurring while giving an insightful narrative into its processing. In case you are interested: https://x.com/i/grok/share/80kOa4MI6uJiplJvgQ2FkNnzP
LLMs will write out explanations that are entirely post-hoc:
> Strikingly, Claude seems to be unaware of the sophisticated "mental math" strategies that it learned during training. If you ask how it figured out that 36+59 is 95, it describes the standard algorithm involving carrying the 1. This may reflect the fact that the model learns to explain math by simulating explanations written by people, but that it has to learn to do math "in its head" directly, without any such hints, and develops its own internal strategies to do so.
https://www.anthropic.com/news/tracing-thoughts-language-mod...
It seems to be about as useful as asking a person how their hippocampus works: they might be able to make something up, or repeat a vaguely remembered bit of neuroscience, but they don't actually have access to their own hippocampus' internal workings, so if they're correct it's by accident.