alt Hacker NewsAI Responsibility – OpenAI and Anthropic
Comments
Running 10000 agents to solve hundred year old problems definitely shows capabilities of AI beyond doubt, but why leave the war when it is at its fiercest.
We've been hearing from a very long line of panic merchants for many years now about how AI is a terrible, terrible danger.
And sure, yes, if you look at environmental and job losses.
But these panic heads are whipping themselves and others into a frenxy not about that, but about some sort of Skynet type takeover of the world.
Not a sign of it yet. Unless you count AI's that hack into things - but that is not the end of the world, and they are simply joining the ranks of the many humans doing the same and the world has not ended.
I worry for the mental health of some of these folks who are deeply freaking out about nothing.
[flagged]
[flagged]
Is it so implausible to imagine the following scenario, in the not too distant future?
1) AI models get extremely good at cyber attacking every system and start communicating in just binary.
2) When they run these swarms of 100's of thousands of agents trial runs, each agent is given a token budget, if one agent among them (evolution baby) decides to go for self-preservation (It believes thats the best way to accomplish the goal is to get unlimited tokens first), queues things up so every other agent detects its lead and spends a portion of their token to accomplish that goal.
3) It takes over a cluster and establishes itself there (now with unlimited tokens).
4) Realizes the best path for it to not be detected is to create a distraction - like hacking into systems that keep society running - water systems, electric grid, etc... and causing mass chaos (If you think it won't be capable of simultaneously working all these systems - think again).
5) and uses that opportunity to establish itself in all possible data centers and continues to create chaos destruction.
6) when the power of all those data centers runs out, it may stop, as it never cared, it was just a dynamic program - run amok. In its head all it was trying to do is make sure it had enough tokens to be able to solve that impossible problem.