alt Hacker NewsHow is it solved? LLMs cannot think new things, they can only cobble something together if it's in their training set.
Replies
What? This is a massive misunderstanding. It’s easy to get truly novel ideas from LLMs, unless your definition of “new things” is so strict that no human can do so either.
The training set is about patterns, not facts or specific configurations. Yes, it’s possible to extract (some) of the training set verbatim, but that doesn’t mean it’s all you can do.
I am not sure if claude had powerpc scripts in its training.
That "only" part used to be the hardest. Getting the ideas was never the hard part. I think someone here even wrote an essay on that.
It does not think at all. It vibes based on its training and any additional bolted on constraints. It is a quite simple automation that only works by huge amount of existing data.
Modern man has grown quite dumb. He only seems to be able to "invent" by massive scaling things that are decades or centuries old..
New things are made by cobbling together existing things.