alt Hacker NewsIn a video of Hank Green, he interviewed an AI expert (if there even is such a thing), and he said that in the thinking part of the conversation, LLMs seems to use code language to communicate with itself, like in the usage and ordering of words, and such.
I think that there could be even more then a fingerprint in those messages.
It’s this video: https://youtu.be/5CKuiuc5cJM?is=9VQ1FCxY_X3eNm-b
Warning: They anthropomorphize a lot in this video, but I get it… the words exist, why not use them.
Replies
He’s not an expert in anybody’s estimation other than his own. He’s a blogger, tech booster, executive, SEO guy, etc. He was big into NFTs. He gloms onto every buzzy tech thing because squeezing all the “gee whiz” he can out of tech optimism is that guy’s career. He has no education in ML or anything related.
I'll take this opportunity to repeat that the natural language interpretation of thinking traces don't appear to be "real" by any reasonable definition. Even if they can at times be useful (at least seemingly). There's research demonstrating the usage of arbitrary symbols, even just repeating a single symbol, leading to a similar improvement in ability. This makes sense if you consider how the attention mechanism and KV cache work as the sequence iteratively grows.
Basically we optimize the models to produce output with certain characteristics but that doesn't mean that what we see is the whole truth or even that the relationships in the underlying system are structured in the way that we might expect.