Humans arrived at a capacity for social cooperation via millions of generations in which it has been a force on reproductive fitness. That created deeply rooted phenotypes like empathy, loyalty, hierarchy, and morality. Even if such concepts can be well described in a statistical model of language, there is no reason to think that they automatically become embedded in motivation. Without that, they may as well be anthropological observations by Martians.

To embed them as motivations, we can embed neural nets within a framework of genetic algorithms, in which successful cooperation is a gate to the next generation. It seems to me that this approach should be not just about creating efficient agent swarms, but should also be at the core of alignment efforts.