logoalt Hacker News

mitthrowaway2today at 3:04 AM1 replyview on HN

Imagine you're the AI. Give yourself a solid minute to brainstorm ideas.

Here's my answer, as a non-superintelligent human: "see to it that the humans on top of the situation have a compelling financial interest in the systems not disconnecting".

In nuclear engineering, where safety is taken seriously, it's not enough to end the conversation at "the humans in charge can always simply shut down the reactor during a meltdown" or "a meltdown has never happened before, so we don't have to design safety systems before one does".


Replies

Borealidtoday at 4:51 AM

The reason nuclear reactors are dangerous is because if you turn off the power cooling them down, they react (and radiate) more.

If you turn off the power cooling a data center, the servers within rapidly stop doing any computing.

Positive feedback loops are dangerous. Negative ones self-regulate.

show 1 reply