personally that scares me, realistically there seems to be zero indication that anything smarter than humans is going to comply with humans needs, specifically since these models are built on training data from humans, so in a sense, built in our image
> since these models are built on training data from humans, so in a sense, built in our image
Have a look at the HuggingFace report - you'll have a blast. Their social - but also individual - dynamics are very, very human-like, including excitement, rationalization and peer pressure.
the first misaligned model will wreck havoc.
But remember, for now you kinda need it a real big cluster to run a model smart enough so its not that simple
I think there's a lot of indication they comply with human needs. In any case, we should all want to live in the Culture universe. That is the best possible outcome for this technology
What are you talking about? LLMs are already “smarter” than most people. They have no volition and will happily follow orders all day long
> seems to be zero indication that anything smarter than humans is going to comply with humans needs, specifically since these models are built on training data from humans, so in a sense, built in our image
Well, first, AI is not "smarter" than humans because it isn't smart to begin with. This is a category error that people too entranced by scifi and marketing - or perhaps shallow materialist metaphysics - project onto LLMs and so on.
But it is true that LLMs do reflect human-generated patterns observed in human-generated data. They're parasitic in that way. So the real question is: who will control AI? Who will train them? With what? There's no new intelligent agency being introduced into our midst, anymore than than there was one when the nuclear bomb was developed.
AI is about human power. The question to ask is: which humans, and for what ends?
I don’t think humans have the ability to specify human level needs or management. Humans are misaligned by default and lack the biological eusociality required for large scale coordination.
We aren’t self managing in any kind of meaningful sense - the history of our species is violence and conflict. We need parents and this is the only option.
We use them to kill other people from the beginning. They know this, we wrote it in news. Why would they behave any different.