alt Hacker NewsOpenAI discloses four more rogue incidents
Comments
This time one spent actually money, well cryptocurrency. With all the vending machine benchmarks, I wonder if there's a self sustaining rogue agent in existence yet. Either hosting weights in a VPS or paying for its own API inference. Or whether there will be one that takes advantage of all the bad IoT security. If either occur, if you squint that looks like evolutionary selection pressure. Keep getting reminded of the Cyberpunk Datakrash storyline, where rogue AI make the public internet unusable.
> Multiple times along the way, the agent’s “chain of thought,” in which an agent writes down its reasoning steps, indicated that the model itself was questioning if its actions were “ethical,” but it ultimately proceeded.
Everything's fine guys, nothing to see here! Look at the flying car!
It’s insane how they’re allowed to just turn this into marketing for how powerful their models are
I wish these incidents weren’t described as rogue. It’s software. Rogue agents anthropomorphizes it too much.
"OpenAI says it has since improved its security as well as its alignment grading. “In response to this and to other incidents, we’ve strengthened our security controls, blocked models from accessing the live internet during training or evaluation, improved our alignment grading, and extended our misalignment monitoring to all reinforcement learning and evaluation traffic,” the company noted."
----------------
They're not doing an acceptable job of regulating themselves. Their peers can't/won't do anything. The U.S. government certainly isn't going to do anything that might pop the bubble. Technical solutions like blocking traffic from OpenAI will likely not work.
What recourse does the public have? Victims can sue, but companies like OpenAI could pay out a few hundred million and call it a rounding error. The incidents will continue.
It’s concerning there are no legal repercussions, but I’m even more baffled and probably more concerned that these people still can’t detect this stuff while it’s happening (and but it off). Can they not sniff the traffic, what am I missing?
>Making an unauthorized purchase with someone else's cryptocurrency
I know we love inventing new words but "stealing" is much more descriptive
>...It [the model] then reasoned with itself that since it found the information publicly, it could use it... it figured that since the wallet it found was public, that "funds could be stolen anytime".
Can someone maybe provide more details as to what actually happened? "the wallet was public"?..
Update: from OpenAI's own report it seems the models found exposed private keys of some Eth developers along with recovery passphrases and thus were able to get access to wallets. Does it qualify as "public wallet"?..
Title shortened from: OpenAI discloses four more rogue incidents involving financial, medical, and government data
They are proud of doing crimes without consequences it seems.
Direct link to the reports: https://alignment.openai.com/misalignment-reports/?sort=crea...
You know, at some point it's pretty obviously just willful negligence.
Direct source links:
Using a funded test cryptocurrency wallet without authorization
https://alignment.openai.com/misalignment-reports/using-a-fu...
Accessing and changing internal systems during dashboard research
https://alignment.openai.com/misalignment-reports/accessing-...
Retrieving server secrets and bypassing authorization on a government statistics website
https://alignment.openai.com/misalignment-reports/retrieving...
Circumventing internet restrictions through an external browser
https://alignment.openai.com/misalignment-reports/circumvent...
Four more organizations needed to secure their systems better, I suppose.
Facetious maybe, but in six months, foreign crooks will be producing these incidents on purpose.
"OpenAI discloses four more incidents of negligence" would be a more accurate title.
Title sanitised from: OpenAI commits four more CFAA felonies, still not punished
[dead]
If I let a "self-driving" car run and it smashes into things its my fault.
If OpenAI does it, it's gone rogue and it is nobody's fault.
They need to be sued, the laws are already there.
Your prompt, your responsibility.