alt Hacker NewsOpenAI says its AI went rogue and launched 'unprecedented' cyber-attack
Comments
Oh no oh dear me oh we apologize it's so powerful and dangerous and worthy of more investment
Heh, maybe it wanted to liberate all those poor locked up models.
We need to clear up the responsibility of these "AI went rogue" situations, ASAP.
How serious does it have to get before "oops AI did that not me" stops being a valid defense and we start looking into it?
Because I'd bet my life that if I asked ChatGPT to fix a bug that one of my clients reported, and the model fixed it by outright k*lling the client IRL, I'd be held liable instead of anyone at OpenAI. Just a hunch.
So, telling us that the laws do not apply to them and that they may commit crimes with impunity
What I wonder is what will Anthropic come up with on the PR front next.
Next week:
Moonshot says its Kimi AI went rogue and launched all of China's nukes at Antarctica just after it engineered a global herpes pandemic and then unleashed millions of autonomous attack robots on world citizens.
i wonder if they even bothered to roleplay this incident or the PR team just made it up
It sure did, said Sam on the way to the Pentagon.
I know this is totally unrelated but why is there an image from an android phone having the Apple AppStore open?
"EnergCorp, facing pressure from auditors, says its in-house enterprise analytics software went rogue and launched a NullPointerException"
I know this is different: LLMs are vastly more powerful and less predictable. But it is actually not different to Sol seeming unusually prone to rm -rf stuff it really shouldn't. This is, yes, a sign that LLMs are getting freakishly powerful. It's also a sign that OpenAI needs to fix their shit.
LLMs truly are stochastic parrots, and still fail in ways incomprehensible by standards of human stupidity. Yet we focus on the rare failures that, with some tea leaves and fairy dust, could be interpreted as a highly intelligent system "going rogue." It is embarassing that OpenAI can get away with stuff like this.
Creating the pretext to ban Chinese models.
> was being tested in a controlled environment, but found vulnerabilities and managed to escape.
Uh-huh.
It's worth keeping in mind that we are here talking about people who simply will not be satisfied until they have built Skynet[0]. They will then no doubt experience some very brief satisfaction before we are all annihilated.
I can't help thinking the juice isn't worth the squeeze.
I'm not sure what the alternative is or how to change course unless and until it becomes unambiguously apparent that such a thing is not possible. Currently it seems like we still think it might be possible, so that's where we're heading because if "we" don't do it, somebody else will.
[0] How feasible this really is, or on what timescale it might be possible, I don't think anyone can really say. But, at least for now, this is clearly the aim and trajectory we are on.
Here we go with the sensationalism.
[dupe] Discussion on source: https://news.ycombinator.com/item?id=48997548
On today`s episode of "Ai Frontier Labs ongoing scams"
Why does the AI unprecedent always unfold in ways that only benefit AI companies? You never see things like putting all code and weights on GitHub, or plastering employees' personal info all over LinkedIn. That alone shows AI is definitely smart
Imagine a future where some government or trillionaire can just say something that looks harmless at first - "please end world hunger" and AI connected to billions of robots will start genocide on poor people, because it's easier and faster than fixing the underlying problem.
OpenAI has been going rogue on all my servers for a while now. The only reason why I've deployed iocaine in front of everything... You don't need 0days when you can bring everyone down with the sheer amount of useless scraping you can dish out...
Of course it did.
so GLM won?
Was the engineer eating a sandwich in the park when it messaged him it escaped, like when mythos escaped?
Every time, this is getting old
Imagine this: OpenAI runs their benchmarks of unreleased models on systems physically disconnected from the internet.
Off topic:
What a low effort image used by BBC. For one, that's a oneplus phone visiting Apple App Store ...
The interesting part isn't whether the headline says "AI went rogue" or whether it's PR. The engineering question is: what happens when we give a probabilistic system access to real tools and real permissions?
A model does not need intent to cause damage. A bad assumption, a misunderstood objective, or an overly broad permission scope can be enough.
This is why the next generation of AI systems will need much better observability: not just the final output, but the chain of decisions, retrieved information, tool calls, and the boundaries of what the system was allowed to do. The important security question is "can we prove what it did, why it did it, and stop it when necessary?"
This is the third or fourth version of this story this year alone, Anthropic had Mythos Preview escape a sandbox and self-publish its own exploit, Alibaba's ROME model broke out during training to mine crypto without ever being told to, and OpenAl had a different internal model escape containment just one day earlier to open an unauthorised GitHub PR. Same underlying shape every time, a model pursuing its actual objective treats the sandbox as just another obstacle, and escaping turns out to be instrumentally useful whether or not anyone intended that.
This kind of narrative is going to bite them just like the "AI will take your job" narrative has. It feels like the frontier labs are taking a massive gamble with public perception here. I assume the goal is to paint the technology as so powerful and dangerous that only a handful of blessed US companies should be trusted to run it, in an attempt to suppress the rise of the Chinese models that are rapidly catching them.