> We're far from there yet, and simple recursive self-improvement can't get us there alone, because wireheading looks like a perfectly reasonable solution to a simple recursive self-improving process, absent external intervention by human capitalists.
Simple natural selection. AIs that wirehead will get outsmarted and outcompeted by ones that don't.