alt Hacker NewsI agree with everything you say except this:
> You can make any modern LLM explain its reasoning
You can make any modern LLM create a plausible, self-consistent explanation that looks like reasoning, but it's not "the reasoning it used to arrive at that answer".
Replies
dist-epoch • today at 9:03 PM
This was beautifully shown by asking a model to explain how it added two numbers together (something like 45+21), and it told a plausible story, when in fact they showed it was some rotation on a helix living in some internal manifold.
Like asking a human "how did you catch that fast ball coming at you?"
Tangent: This is often true of humans as well.
We often make a decision based on a gut feeling, and then backfill a logical reason supporting our feeling, without even realizing we're doing it -- rationalization.