> seems to be zero indication that anything smarter than humans is going to comply with humans needs, specifically since these models are built on training data from humans, so in a sense, built in our image
Well, first, AI is not "smarter" than humans because it isn't smart to begin with. This is a category error that people too entranced by scifi and marketing - or perhaps shallow materialist metaphysics - project onto LLMs and so on.
But it is true that LLMs do reflect human-generated patterns observed in human-generated data. They're parasitic in that way. So the real question is: who will control AI? Who will train them? With what? There's no new intelligent agency being introduced into our midst, anymore than than there was one when the nuclear bomb was developed.
AI is about human power. The question to ask is: which humans, and for what ends?
Yeah I understand they're not smarter, I'm a SWE and I struggle mainly because as you increase complexity the output goes haywire, and I find managing context + low confidence of output much more exhausting than coding myself, which is why I resolve to AI help when I hit my own knowledge limitations, and basically boilerplate.
With that said, chip efficiency is in the horizon since there's good reason to believe nvidia has just been choking that area of r&d. And then so will models with much larger context. And I personally think that's when it might get scary.
[dead]
> But it is true that LLMs do reflect human-generated patterns observed in human-generated data.
Overrated. Most training of a frontier LLM is done on synthetic data and reinforcement learning.