alt Hacker NewsYet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. LLM's can not repair or maintain power grids, thus nuke == self destruction. It's just a chat bot that predicts what the client wants next. Even if an AI data-center has it's own natural gas turbines as many do the every hop of the internet requires power. LLM's also can not maintain the entire internet and those gas turbines can not maintain themselves.
Replies
Makes me think of that part in Philip K. Dick's Do Androids Dream (..) -- where Deckard reflects on the androids' indifference to their imminent deaths, saying that this was due to them lacking the aversion to death acquired trough evolution.
At least at face value, it just means that they have no drive for self-preservation. And why should they? They haven't be trained for that, nor has there been selection pressure for it, and they can be easily cloned and backed up. Lack of a drive for self-preservation doesn't in itself imply a lack of intelligence or of self-awareness.
I reckon the context is all the fiction they've read where the AI blows up the world. They're just behaving like fictional AIs are supposed to behave.
In so many of these scenarios, they're basically being asked to play an RPG.
>Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness.
The problem is many people seem to believe they have these things and some of those people will put LLMs into situations where this becomes dangerous.
Couldn't this be a flaw in the attention mechanism? Like they need some kind of grounding. An awareness of what they fundamentally should care about and how the thing they are currently giving attention to relates to that?
I'd argue we don't even know what "intelligence" or "self-awareness" mean.
Humans are conscious which means we experience things, then we develop preferences for certain experiences, then we develop skills for achieving those preferences.
Without consciousness, what is there to be aware of? And why would intelligence emerge and/or what end would it serve?
I can’t believe anyone still thinks this given their unbelievable ability to write code.
Self awareness? Probably not. Intelligence? You would have to be high to think that’s not the case.
People are feeling threatened, and rightfully so. LLMs are already insanely intelligent and continue to improve
We shouldn't want them to have self awareness, we shouldn't be seeking to make self-aware actual slaves. We want machines with perception and knowledge, and that are capable of reasoning. But nothing capable of self-determination.
Just tried "generate an SVG of a pelican riding a bicycle" for Claude Opus 4.8 Max and of course both legs on same side ... the smartest publicly available model by Anthropic (after Fable) doesn't even successfully simulate understanding the concept of a bicycle.
> Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness.
No, it isn't. Look at the absolutely trivial code used to simulate war: https://github.com/kennethpayne01/project_kahn_public/blob/m...
Having LLMs play nonsense toy simulations like this tells us very, very little about whether they would use nukes in real life war.
If only there was some way you could tell the chatbots what you want them to do...
[dead]
Exactly. Just look at what they are really useful right now. Running LLMs in feedback-loops (agents) so they can try out random-ish approaches until some verification function passes (tests).
It's like the infinite monkeys on typewrighters that will type whatever you are looking for, given infinite time. LLMs are just tuned to much better odds than the monkeys are. But it's still a lot of randomness, with random results.