alt Hacker NewsIt's a simple prisoner's dilemma scenario. If you focus on safety, you're still exposed to all the risk of extinction when your competitor achieves ASI first, but you lose the upside of potentially becoming king of the world. There is no possibility of future rounds, so the rational strategy is to always defect.
It can certainly be seen as a simple prisoner's dilemma, but it's not in some very important dimensions. (E.g. given the core tech the most likely outcomes are not AGI but developed carelessly nonetheless capable of causing all sorts of societal damage.) Unfortunately our bitwit overlords love short term self serving frameworks like this one so it's easy to imagine them embracing a "what has the future ever done for me" strategy...