logoalt Hacker News

jrflo • today at 5:36 PM • 3 replies • view on HN

I feel like their logic is internally consistent, even if you don't agree with the key assumptions.

1) RSI is very dangerous, and such self improving models should not fall into the wrong hands.

2) Releasing an open model is like opening Pandora's box, you can't recall it once it's released. If that model is capable of RSI, there's nothing you can do to stop bad actors from getting superintelligence.

3) Therefore, the safest thing to do is to ban open weights above a certain capability level. The problem is we won't know what that capability level is until it happens.

I don't know what the "weakening safety pledges" part means, would be curious to learn what they're referring to.


Replies

agnosticmantis • today at 6:03 PM

Are the rogue actors in the US? If yes, then there are existing laws for existing crimes (including cybercrimes). Enforce the laws on crimes not technologies.

If not, and the US law enforcement can't touch the rogue actors, then your US regulation only disadvantages good actors in the US (and hugely advantages anthropic et al by killing the competition ofc), and the rogue actors get their open models from China or whatever anyway.

➕ show 1 reply
darepublic • today at 5:44 PM

4) AI is capable of having a soul. With all the new souls that will be released into the world, Dario will be their pope and his captains the high priests

Aperocky • today at 5:55 PM

4) Only our model is safe because we are us and Dario believes in "Effective Altruism", trust us!