alt Hacker NewsIn this article there is an implied step in which, when AI becomes ASI, then as a natural consequence it is suddenly in control over humanity. How should this play out? It hacks all nuclear launch sites and starts to make demands that we are forced to follow? Takes control overnight of all computers on the planet and locks humans outside? Is that even possible? Even so can't it be stopped by turning off datacenters and network connections? How does it take control of all the airgapped facilities (which would be the most "useful" in a takeover)?
I think it's too big of a question to leave it as an unexplained implication.
Also in the not/ASI scenarios, they imply a worldwide adoption in a short time. The world is still adopting inventions such as the internet, email or even electricity in some parts of the world. Why should they instead adopt AI overnight?
Germany, third largest economy in the world and largest economy in Europe, is still very much not digitalized. The whole country runs mostly on paper, hand signed contracts and notifications sent by post. As a country, they still have to adapt to email and the internet. Japan is similar for what I heard.
I find this article good food for thought, but it misses some crucial pieces.
Replies
as with all doomers, they go from AI one day to ASI the next, never talks all the moves/counter moves humans make to get there. Is a canary when ever I see that.
Pretty much the only outcome that isn't Yudkowskian scifi is Enduring Complementarity. I'm not an AI hater by any means, but can AI enthusiasts get more representation that isn't "the world as we know it ends tomorrow"?
> In this article there is an implied step in which, when AI becomes ASI, then as a natural consequence it is suddenly in control over humanity. How should this play out?
The real world isn't a game of chess, we do not know all the moves that can be made. In addition to the narrow scenarios you've suggested, I can also easily imagine two distinct variations of "AI talks people into letting it take control" (look how often control is already voluntary given up for convenience), plus "drugs us" (exogenous or endogenous), and "is this before or after Musk convinces a lot of people to get those brain implants he specifically marketed as how to merge with the coming AI super-intelligence?".
> Takes control overnight of all computers on the planet and locks humans outside? Is that even possible?
Between (1) vibe coding, (2) how effective LLMs already are at finding vulnerabilities, and (3) how LLM swarms already sometimes demonstrate self-sacrifice for other LLMs?
Yes, this is possible.
> Even so can't it be stopped by turning off datacenters and network connections?
Theoretically. Three problems: (1) the economic cost of losing all this would be enormous, so people would ask "are you sure?" a lot. Look at how little has been done about climate change due to the cost: from observed human behaviour, I think humans will stall a lot here.
(2) Coordination. How do you do this everywhere when you've got no control over the computers?
(3) How do you even find out? LLMs can fake text, deep fakes were already a thing before diffusion models and other GenAI image and video tools. None of those are perfect now, yet despite the flaws they're still capable of fooling people during live video calls and getting them to make catastrophically large financial payments out of their companies.
> How does it take control of all the airgapped facilities (which would be the most "useful" in a takeover)?
Lots of supposedly airgapped systems have flaws. Quite often, the easiest flaw to exploit is a human worker, whom you can get to plug something in that they shouldn't, which already happens a lot.
> The whole country runs mostly on paper, hand signed contracts and notifications sent by post. As a country, they still have to adapt to email and the internet.
Yes, many jokes about this locally. Yet even here, lots of digital stuff. But let's ignore all that: how effective would a coalition of non-digital economies fare against all the digital economies being coordinated against them?