alt Hacker NewsI like this anecdote because it gets at how words to an LLM have no connection to their real concepts. To an LLM, words are simply numbers arranged in a likely pattern.
I like this anecdote because it gets at how words to an LLM have no connection to their real concepts. To an LLM, words are simply numbers arranged in a likely pattern.
Of course for humans words have no inherent meaning either, they're just sequences of characters or patterns of sounds. It is what words are associated with that carries meaning. A large part of this is how words relate to other words. LLMs can capture this in principle. What LLMs lack is the direct association of a word with sensory experience. But it's an open question how relevant this is in practice to understanding.