alt Hacker NewsI always wondered what the model meant when it writes "I'm now considering the architecture of the service" but outputs nothing of the sorts in its CoT.
Is the model really "thinking" about that stuff or is just mimicking human "manners"? And if so, where the thinking is happening if it is not in the literal chain of *thought*?
I'm not sure J-Space is the answer to that question, but very interesting nevertheless.
Replies
> Is the model really "thinking" about that stuff or is just mimicking human "manners"?
Well, what's the difference? If it's pretending to think and its thoughts correlate to its final output, then I'd say that really is thinking.
Almost none of the hosted models give you their unredacted CoT. Claude certainly doesn't, what you get are fragments and summaries from it.
There are various justifications on this, but it's mostly to make distillation and fine tuning off their model outputs a bit harder for their competitors
> I'm now considering the architecture of the service
What you see here is a summary of thinking tokens written by some other smaller model (e.g. old sonnet). The actual thinking sometimes (rarely) leaks and is not easy to parse.