alt Hacker NewsBreaking: language model whose purpose is to predict the most likely token, after being trained on non-uniform human-generated dataset, does not follow a uniform distribution.
Replies
singpolyma3 • today at 12:22 PM
What's interesting is not that it isn't random. But rather the particular way in which it isn't random.
IAmGraydon • today at 12:39 PM
Yeah I have no idea why anyone considers this interesting. More evidence that most people have no idea how LLMs work.
People are also not remotely random in this respect.
See e.g. the "blue 7" phenomenon [1]. While it is disputed by some, I've personally witnessed it "second hand". E.g. before learning of it (I was aware of the general principles of cold reading relying on stats and knowledge of human nature, but not how to do this particular one), a former boss of mine came back from lunch all excited and recounted a guy who'd run a cold reading routine on him that involved the guy getting him to think about blue and 7. Before he got to the answer, I already knew the answer was going to be blue and 7.
[1] https://en.wikipedia.org/wiki/Blue%E2%80%93seven_phenomenon