> Most of my criticism of Ben’s perspective is against the idea that some kind of emergent morality that we would recognize

I think Anthropic has already provided some evidence that intelligence is tied to morality (and vice versa) [1]. When they tried to steer LLM models morals they saw intelligence degradation also.

[1]: https://www.anthropic.com/research/evaluating-feature-steeri...