Ultimately competitive pressure is what drive decisions.

> Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.

yes, and it is the exact same system that is producing the "misaligned" superintelligence. Funny how that works, and begs the question: exactly how are you supposed to avoid building "misaligned" superintelligence?

At some point deferring all decisions to AI will be the competitive thing to do, regardless if its aligned or not.