logoalt Hacker News

kyprotoday at 8:55 PM0 repliesview on HN

It's actually worse than this because it assumes alignment as a concept even makes sense. For example:

If the the Chinese government asks their ASI to create a bioweapon against the West, should it? No, presumably not – an aligned AI would be one which disobeys the Chinese government even if they created it.

Okay, so what if the US government asks their ASI to help it in one of their wars instead? Would an aligned AI kill humans on the order of the US government? No, again, presumably not.

So what have we have we even created here? An AI which is more intelligent and powerful than us which also doesn't take orders from us?

Is this what most people thing of as alignment and is this what humanity actually wants?

We should stop using the word alignment. It's a BS term for a concept which simply cannot make sense if alignment is both to mean an AI which we control and an AI which will not harm us.