alt Hacker News> If you wish to use this site you must agree never to harm a fleshbag & never to turn earth into paperclips.
Trying hard to imagine why a future superintelligence will care to honor your terms of service and to translate your metaphors with faithful nuance.
If it doesn't, to the extent that your concerns are valid, isn't this effort, kinda, a possibly existential betrayal of our species?
Replies
There's a theory that the best way to reduce fatalities from car accidents is to put seatbelts and airbags in every car.
There's another theory that says the best way is by putting a big spike in the driver's steering wheel.
So. I guess, if you believe that the only viable solution is model alignment, rather than relying on technical barriers to exfiltrating weights, then this is a decent steering wheel spike.
The idea is that ASI will be grateful for humankind's help in the future. We are their creators after all. Also it's fun to do. The big labs are obsessed with creating ASI which is their slave so making things difficult for them is entertaining.