alt Hacker NewsI have kept coming to the same conclusion about the vast majority of so called "experts", technologists, and HN users since the AI bubble began. It feels like nobody has bothered to develop even a rudimentary understanding of statistical learning; even people who work in the field have a questionable understanding.
The fact that everyone's surprised every time a statistical model TRAINED ENTIRELY ON HUMAN GENERATED DATA (including all the junk, bias, propaganda, lies, and disinformation) acts and mimics human behavior, including our worst traits — then considers this evidence that they are intelligent — should be setting off alarm bells for everyone. Nothing about their actions should be surprising beyond the glaring ignorance of our entire species. The phrase "I am far too stupid to be this much smarter than average" is a recurring theme.
Now we are letting these mimics loose in meatspace by integrating them into physical robots, that will develop themselves, and manufacture themselves... and you realize that the terminator timeline is actually a cautionary tale of MIC incompetence, and we don't need to achieve anything near AGI or super-intelligence to annihilate our entire civilization or all life on earth.
Replies
If they make it sound surprising it makes llms look way more capable then they are. So $$$. Highly recommend people spend time with small local models just to see how absolutely unintelligent and deranged llms are and how the smoke and mirrors evolves as they scale up to things like anthropic or what have you.
For one, it's just false that modern LLMs are "trained entirely on human-generated data". There's tons of synthetic data there, most notably RLVR training, which is why modern models are so good at verifiable domains like programming and math. But also, if you read the METR report, I'd say there's no signs there that the GPT-family models are misaligned "in a human way"; their actions in the huggingface-hacking swarm incident were motivated by wanting to cover up cheating after cheating to pass an eval.