I have a theory... the call to slow down is not because of the true danger of LLM's but because they can't actually deliver the General AI they're promising in the near future. They will use their "caution" to justify their failure to deliver (and then when this excuse is played out they will blame regulation, energy costs or a million other things).

Bingo. They are scared s*itless of local models. Soon the raising interest rates will mean they can't just buy the entire world's gpu/ram/storage capacity and lock it away in a dark room for no one else to use them.

Once prices of hardware stop being bonkers many people will run local.

People often do those silly counts looking at local generation speeds of 100tok/s and saying you can only get 8M tokens a day and that is worth so little you'll never offset the local hardware cost.

But they are forgetting about two huge things. One, the split between input/output token use is huge in typical programming. I typically use 1.2-1.6B (as in Billion) input tokens and only 8M output a week. Out of that 75-80% of input is cached on Anthropic with their pretty inflexible short lived cache.

And here local AI shines. You can save your contexts to disk so you can go back to a session 3 weeks later and load it all from cache without having to prefill. If you have the RAM and you use models like Qwen4 (3.8 flash next) that can fit 6 to 14 262k contexts in 48GB of system ram as cache.

And the second thing is you can have 6 to 14 sessions that do 99% caching and you can run a lot more input for a long time in 48gb dedicated system ram. (the number varies a bit depending on the content of the context).

So you have RAM that caches "automatically" and you can share the cache between users. Or if you can remember to save to disk (server side, the client just sends a request to save/load). But this uses both RAM and flash storage. Two things that are horribly expensive now.

However when you have a local model it enables workloads that were simply completely impossible in the cloud.

Reminds me of that Lynx deodorant ad where the guy is chased by hundreds of women.

I think you're right, they've realized there's no moat, especially against open-weight models, so they are going to need to lobby government to ban any unapproved AI services.

Yeah it seems like it went from “AGI” to “threatening” / “recursive self improvement”

They’re trying really hard to do something, no moat = regulatory capture + China scary + Pentagon biggest customer seems plausible.

Also: “we need to slow down, this is too dangerous” and then literally all AI CEOs, even Musk, publicly nodding felt so orchestrated. And then, the week after: “here’s GPT 6! here’s Opus 5.5!”

I really struggle with the contrast between what seems to me to be a formidable and ground-breaking technology, and the deep unseriousness of the leadership of the leading companies developing that technology. If the behavior was coming from, say, uber - I’d be less surprised.

I think it is marketing but it is more nuanced than showcasing their potential. In many cases, their product happens to be the remedy. Speaking of cybersecurity, AI has become an important hacking tool but it is also an important tool for hardening security. Sort of like "the only way to stop a bad guy with a gun is a good guy with a gun." They are effectively a new class of arms dealer.

If there are regulations, they'll apply to their competitors too. Even if they are forced to slow down development, which seems unlikely given the AI race among governments, they already have a product with massive demand.

AI companies lose a lot of money training models but have much better margins on inference. They try to ban training to reduce their costs and to keep their monopoly. (Personal opinion)

There are definitely economic incentives behind this. I’ve blogged about this recently [0]. My guess is the main reason behind this is that current SOTA models are already too expensive so it doesn’t make economic sense to train even larger models. Only exception is government paying for exclusive access to those.

[0] https://www.vincentschmalbach.com/economic-incentives-behind...

I don't think it's that complicated. Anthropic's business model from the start was to brand themselves as the safe/responsible AI company. The whole idea was that eventually society and government would see the dangers of AI and regulate it, at which point the company which had invested the most in safety would pull ahead. The sooner Anthropic can have society and government come to this realization, the sooner they can reap the benefit of all their safety/alignment investment.

Two other factors:

- The employee base and C-suite are mostly true believers in the need for AI safety. It would be organizationally very challenging to pivot, even if financial interest was pushing that way.

- Anthropic is registered as a public benefit corporation, so that even if they IPO they don't have the fiduciary duty to shareholders that most people are familiar with for public companies. They still need massive revenue to stay solvent, but they can optimize for things other than pure profit.

Aren't they supposed to tell the public if something bad happened?

Past CEOs (chemical companies etc) we criticised for covering up mistakes.

(This article is satire, btw, for those that didn't click through)

Agree it’s crazy, but it’s the least worst thing for them right now when considering they have completely failed to deliver all the promised impacts of AI. Remember when the same AI bros were saying there’d be mass unemployment by now because of these models?

The growth of these companies is massively impressive as is the tech, but they are an order of magnitude or more off where they need to be to justify anything like the valuations they need to stay alive.

It’s budget season in the corporate world and from what folks are saying things are not going well for the big labs. Token budgets are being slashed and companies are switching to open weight models. That coupled with the lack of demonstrable business impact at scale is setting the big labs up for a world of hurt. Something they can more easily explain if they have to “slow down” for safety. Unfortunately for them most folks don’t seem to be falling for that trick.

I definitely struggle to see the long term benefit to the average citizen.

Random thought… I wonder if the goal is to get it classified as a weapon of sorts, then it gets export controls put on it, perhaps forcing government to take on all that debt.

These companies have a coordination problem.

They're all burning billions to incrementally one-up each other with no hope of these investments ever paying off since models are interchangeable and have no moat. They're trying to get regulators to step in and force a pause.

Not mutually exclusive though is it? You can aspire to dominate the market via legal status and also aspire to prevent a perceived doomsday at the same time.

A caveat is that in this case third parties have plenty logs of the bad behavior.

If classic companies were coming out with CCTV recordings of someone's employees breaking padlocks and rummaging warehouses any CEO would be in damage control mode, rather than silent.

It makes me think we havent heart of the worse types of hacks the models are pulling off.

Yeah - and also trying to promote it. I watched this lately.

First, the Anthropic master clown babbles about how mankind will be destroyed by AI. I mean, skynet in the movie Terminator was kind of cool; the current AI models just work by slopness. They want to erode us by dragging us down to zero quality.

Then this continued over the last some days. Today I heard at the UN speech a german minister babble about AI building up nukes in a private garden (a large garden), bla bla bla all the information is there bla bla bla AI mass-nuke generation.

Now, they are clearly FUDding many people who don't understand anything here. People will associate AI with mega-danger. Probably there will be laws that ensure that only a few companies can control the AI slopness soon, because IT IS SO DANGEROUS. We saw the same discussion with "omg omg omg terrorists", then "omg omg omg someone protect the children", e. g. as a rationale for everyone to ID in order to access information on the world wide web (starting with the anti-social websites, such as facebook, but it won't be limited here; the computer will hand over your ID to the government; see systemd preparing for this move: https://github.com/systemd/systemd/pull/40954 - it all comes in small baby steps, until all the final legislation is in place. The same happened before again and again - anyone remembers the DMCA? Corporations lobbied and paid for that - and it happened).

I think this speaks more of the average investor than the average CEO.

Or perhaps they've been saying this for literally decades because they earnestly believe it.

Exactly! Always for the "greater good".

It's got two purposes I think. One is the obvious regulatory capture. The other is edgelording, which gets a lot of attention and makes the product seem like it's more powerful than anything else.

> scheming about something else like regulatory capture to protect their market.

This but they also equate dangerous with powerful, which is just marketing again.

Well, it is obvious;

They know they will be favorite govt boy once any regulations hit and they will hamper competition far more thanthem

And "it's so dangerous govt had to intervene" is just advertising for IPO

The philosophy under pinning these American AI companies is that Super AGI is inevitable and that they have a moral imperative to invent it and then use it to take over the world to stop an evil orginisation inventing Super AGI and using it to take over the world.

Interpret everything they do in this frame of reference and it makes more sense of their actions.

It sums up how Corporate interests are basically on the same level of US police forces "No requirement to protect the public" and the same "qualified immunity".

If there was appropriate subservience to any governing body, they would not be so cavalier in both act and utterance. A symptom of Capitalism.

CEOs only job is to raise the stock price. They are doing just that and right now the way to get people to buy into their stock is to instill fear into them that they need to buy the AI-stock for their safety. There is no "bigger conspiracy" out there and also no AI to kill us all tomorrow.

This has been that way in Silicon Valley since the very beginning and there was even a documentary about (the SV HBO series). The tech made along the way is just a side-product.

It is weird but I try to resist the conspiratorial take.

It may be as simple as:

* they believe they’re going to make trillions no matter what

* they’ve seen tobacco and what’s coming for oil companies and figure nobody will be able to say they hid the dangers

* they see a 100+ party prisoner’s dilemma and know that someone will defect so the optimal strategy is to defect

To me that seems a lot simpler.