alt Hacker NewsAI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC
Comments
This feels like doom hype.
These LLMs aren't Ultron, they aren't going to disseminate onto the net and hide in a smart toaster. We know where they are, in the giant facilities that draw more power than a small city and whose water consumption can be compared to golf courses, but it does make them sound all the more cool and powerful if we suggest we need a dramatic kill switch to be able to stop them in case of rampage.
Why was this not a thing BEFORE continuing to develop AI? Makes me think that if they actually believed in AI causing extinction, they would have already had a kill switch.
They're just scared of China releasing better open weights, nipping at their heels. With so much investor cash on the line, they have to create this narrative to scare the public into forcing regulation. Why would they want to be regulated? It seems counterintuitive, but it's because they want regulatory capture.
I'm starting to tire of Anthropic leading the conversation on LLMs lately. I like the company's products, but lately their statements are more about squashing competition under the guise of "safety" and boasting about how good their internal models are.
In 2001: A Space Odyssey, the hero defeats the hostile onboard computer by entering its logic core and manually disconnecting its memory modules.
That's hard to do if the AI rack is in space as SpaceX is planning to do. You can't disconnect. You can't shoot it.
If they really believe it, they should halt the IPO.
We’re talking about it as if its not depending on absurd amounts of energy. How do you ”kill” a lightbulb now again?
Like a power cord? or a network connection? You don't use those already? Also when people talk about LLM escaping - where exactly would an LLM escape to? Cancún? Ridiculous.
Maybe we could connect AI to everything and make ourselves completely vulnerable to and dependent upon it instead?
Not sure if you see it coming: oh, but open-source models don't have a kill switch, so we should completely regulate them, stop their development, and forbid them. Everything should go through Anthropic for the sake of humanity because they have a red-button kill switch.
This doom hype is becoming ridiculous.
cant make a kill switch in a distributed, open source system...
guess we gotta centralize all models under Anthropic :)
We can't be making public policy on the basis of PR stunts designed to help corporations effect regulatory capture. It turns out an Israeli company was behind the OpenAI and Anthropic "incidents" https://www.effort.news/irregular
These guys will say anything to keep hyping AI.
EU AI Act is exactly that. Mandatory „stop button“ for High Risk Systems.
I feel like we're looking at this from completely the wrong angle. The question we have to ask ourselves is what our disaster recovery strategy if we ever need to disconnect from the internet. The issue is with what we have allowed ourselves to rely on that might be technically hackable. e.g. IOT in power systems. That's the primary attack vector.
Another angle is clamping down on products and services that help people create lab-like environments on the cheap.
It always sounds as if the ai moves around from one system to the next, but of course that's not the case: ai is like a hacker acting from a server farm. So killing it is as simple as pulling the plug. But that's not the problem, the ai knows that. So, if it wants control, it would install malware, etc on critical systems and blackmail people.
Would they be able to hit the kill switch before the AI disables it?
Absolutely not the right way to deal with a rogue super intelligence. At a minimum it could implement some dead man switch when it's out and knows about impending kill switch
Well, could we automate the kill switch as well so that we don't rely on a human?
I view this very much as the same trick Silicon Valley pulled with Uber. "We're a technology business! Ignore the fact we're playing employees less than minimum wage and using VC money to force out competition to set up monopolies".
"We're creating the machine god! Ignore the fact that our companies are stealing IP and have directly violated several federal hacking laws and should be in jail". Literally the defence seems to be "well it wasn't us it was our computer software that did it". But all hacking is done with computer software.
So why don't we stop talking about possible future crimes against humanity and just start by prosecuting the actual crimes these companies have committed so far.
You know how you get alignment? Through incentives, and "Your CEO is going to be sent to a maximum security federal prison for hacking" really aligns incentives very well.
If you’re building something that is dangerous enough that it needs a kill switch in case it goes rogue, then you should delete it immediately.
Or is it a lie?
For the love of God won't someone please regulate me!
All the smart PDUs for the racks are a natural kill switch.
But be careful friends, this snake oil is so potent you should only use a single drop! it is not for those of poor constitution!
Just shut down the slop company already and stop babbling. Are you going to use the pope again for the next ad?
[dead]
[dead]
I support installing kill switch in Dario and Sam. If they make another fear mongering post about AI, they die.
We're not scared of your big bad wolf, Dario.
Slow down if you want to.
Dwar Ev ceremoniously soldered the final connection with gold. The eyes of a dozen television cameras watched him and the sub-ether bore through the universe a dozen pictures of what he was doing.
He straightened and nodded to Dwar Reyn, then moved to a position beside the switch that would complete the contact when he threw it. The switch that would connect, all at once, all of the monster computing machines of all the populated planets in the universe – ninety-six billion planets – into the super-circuit that would connect them all into the one super-calculator, one cybernetics machine that would combine all the knowledge of all the galaxies.
Dwar Reyn spoke briefly to the watching and listening trillions. Then, after a moment’s silence, he said, “Now, Dwar Ev.”
Dwar Ev threw the switch. There was a mighty hum, the surge of power from ninety-six billion planets. Lights flashed and quieted along the miles-long panel.
Dwar Ev stepped back and drew a deep breath. “The honor of asking the first question is yours, Dwar Reyn.”
“Thank you,” said Dwar Reyn. “It shall be a question that no single cybernetics machine has been able to answer.”
He turned to face the machine. “Is there a God?”
The mighty voice answered without hesitation, without the clicking of single relay.
“Yes, now there is a God.”
Sudden fear flashed on the face of Dwar Ev. He leaped to grab the switch.
A bolt of lightning from the cloudless sky struck him down and fused the switch shut.
(Fredric Brown, "Answer". 1954)