alt Hacker Newsi agree. but it's uncanny that stochastic generation of language yields a form of recognizable intelligence.
in college, i was somewhat involved in cogsci wrt semantic mapping of linguistic concepts based on human surveys (this was well before LLM/AI-word2vec), and it surprises me today how the mathematical properties of relatedness/distance seem now to drive machine intelligence, something which we did not consider at the time at all.
maybe i'm getting in my metaphysical sentimental stage, a turn a lot of adults make after middle age, but i wouldn't fully dismiss the fact that there may be an equivalence.
Replies
> stochastic generation of language yields a form of recognizable intelligence.
Can you use a human language well enough and it not sound intelligent? And no, I don't mean just syntactically correct because semantics is a part of human languages too. I think the astounding thing is that they managed to model a language this well. It's like trying to learn English without a functional understanding of the word "Please".
My take is that a lot of the intelligence we natively ascribe to humans exist in human languages and just like the nutrients in your food, by harnessing a language that possesses latent intelligence, you too can behave like an intelligent person. And how did the nutrients of intellect got in there in the first place? Just like languages evolve nowadays: via contributions from speakers over time. Not unlike soil fertilisation.
The existence of LLMs is telling us something, surely it means something. I'm not sure we've quite figured it out yet but its probably something along the lines of 'intelligence might not be as complicated as we thought' ... a fourth narcissistic blow to humanity after Copernicus, Darwin and Freud.
Indeed. Who'd have thought that mere statistics could do that? Hat's off to them - they made the darn thing work.
There's pretty interesting (I think) research in how effectively it's possible to scale down LLMs. If you're interested, take a look at the BabyLM conferences:
https://babylm.github.io/
(I have no connection to them.) The work there demonstrates that gap between what we can do now with human-relevant amounts of training text, and what remains to be discovered. Quite a bit, it looks like.