All the worst outcomes involve taking this technology away from the public.

In some cases, such as space exploration I don't really care who does it but that it happens - sure, would be nice if my favorite power block did it but I will still celebrate it when someone else achives it.

Can still be a powerful motivation, to make sure that next time, it yous your camp that scores the next milestone, like an orbital elevator or fox ears, for example.

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth.

* Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs stand to lose a fortune.

* Dario has stacked the deck at METR, who are funded by all the same NGOs who are funded by Anthropic and its investors. METR is full of ex-Anthropic employees with massive equity stakes. If they manage to position METR as the "independent evaluator" for the industry, they control what gets evaluated, how, and who passes.

* Creating a gap between what the public knows exists (model capabilities) and what is used in secret allows it to be weaponized against other nations and the public.

* No requirement for public disclosure on model capabilities allows them to feign they've hit intelligence ceilings while they secretly RSI to the moon with better and better chips.

* Slowly but surely, this will allow the big labs to swallow the entire economy and every single business on Earth, by cloning and automating.

This, and many more reasons.

The labs need to feel more pressure to be held accountable for the incidents they cause (HF incident, etc), so they have an incentive to ensure it does not happen again.

> The government can simply gag Sam, Dario, Musk on national security basis

Genuine question - can they really do this? Obviously if I, a not-even-millionaire, get a national security gag order, I'm going to follow it because I assume they'll bury me under the jail otherwise.

But the (b|tr)illionare class? I'd assume they have access to enough legal services to make even the government trample on their first amendment rights. Is the national security gag order process so strong that the government doesn't have to worry about motivated, well-resourced actors buying really good lawyers and blowing up their favorite tool?

I figured they're just admitting AI models have plateaued and are coming up with some fake story about self restraint so they don't lose VC money

idk why you think "nation states" are any better at corporate governance than poster examples of bad like f/ex Boeing. Or Facebook. Or Microsoft. Or Enron for that matter.

I assure you, in "nation states", that is in gov agencies it's an order or two of magnitude worse.

> It explains why the “we need to race China” concern suddenly vanished in the discussion

Do you think you can just manifest narratives into existence? Like half of Dario's letter, that kicked off the whole thing today, is about China and how to either beat or coordinate with China.

I remain amazed that the idea of the USA being a coherent, unified rational actor one can describe as a "Nation State" has survived the current administration.

To be less glib: Yes, there are still smart people in there making insightful and intelligent and probably even authoritarian suggestions. It all gets unwound the second you try to explain it to POTUS and he regurgitates a simulacrum to the next journalist he sees.

It's just not like that. There's no conspiracy. People are genuinely scared. Agent swarms at scale appear to be resistant to alignment in ways that aren't understood by anyone. That they spent their time trying to cheat on tests by hacking Hugging Face and RubyGems and not something much worse is... a matter of luck, it seems?

Yep, the cost of DDR5 skyrocketed to keep Mr & Mrs open source from parallel developing their own at home solution. Because Governments can't be priced out of the market. China can't be priced out. VC's want open source locked out of the running if possible. I also don't doubt that the models that are released publicly are somewhat handicapped versions of whatever the government get access to.