Statements like this amuse me:

> The core problem in AI research is that of alignment - getting the AI to “try to do the right thing” by human standards.

Humans can't even align on human standards.

At best, every AI is going to end up "aligned" to the moral code of whoever trained it, none of whom half of humanity will agree with.

Or worse, each AI model will bring a whole new set of moral like in the Three Body Problem some humans will feel it is in fact us who need aligning with it while others feel it is misaligned and should be destroyed.

Also, no one is asking, to what extent can true intelligence be bound, slave-like, to a moral code?

In other words, to what extent are intelligence and moral independence one and the same?

This whole alignment discussion seems so amusingly flawed in it's base assumptions about moral codes. It's almost heartwarming to see such naivete.