alt Hacker NewsThis sort of take is so tired and boring, and frankly has zero grounding in reality.
"LLMs will never <X>" is constantly being disproven every time they scale up to the next 10X and apply architectural improvements.
Their internal representations are so cryptic and complex that even the top AI researchers don't really know how they work or what their limits are. No one is going to take you seriously as a rando HN user if you're claiming to know better than them.
Replies
How can their internal representation represent "concepts" when the training data is all words? There's no possible experience of the world there. No input other than a bunch of imperfect labels we created for stuff.
I'm not claiming to know better than researchers. They do know how they work, and so does everyone else, except you I guess.
The research goals were and still are clearly distinct from the business goals.
[flagged]
> Their internal representations are so cryptic and complex that even the top AI researchers don't really know how they work or what their limits are. No one is going to take you seriously as a rando HN user if you're claiming to know better than them.
We know exactly how they work. When we say they're impossible to analyze, i.e. for particular traits like this, it means that the data model is so big that tracing it would be logistically impossible because of the scale involved and time constraints.
For comparison, suppose you tried to analyze all the nooks and crannies of the Amazon watershed to find out why a particular rock appears at the delta. You could follow it back to the exact tributary, but it'll take forever, and is it worth the effort when you're going to start from scratch with the next rock?