As far as I can tell, much of Anthropic genuinely believes that someone will build an AI in the next 3-20 years that's significantly smarter than any human alive. Sounds wild, but a lot of their people have been saying this since 2018 or even earlier. I think they're true believers. Furthermore, they believe that building such an AI would be dangerous.

So their plan is:

1. We can't stop other people from building something dangerous.

2. But we can get there first.

3. If we build it, it has maybe a 15% chance of killing everyone alive. (I think that's a number I've seen Dario use before, but I may be wrong.) If OpenAI or China build it, the odds would be worse.

Obviously, if Anthropic is actually correct about (1) and (3), then nobody should allowed to build frontier AI.

People find it really hard to believe that (a) anyone believes in the possibility of dangerous AI in our lifetimes, and (b) that someone could believe what Anthropic seems to believe and then still go ahead and gamble with everyone's lives anyway.