> However the cold reality for both is that there is zero moat to a model anymore.

The moat right now is a) the hardware, b) the electricity, c) the intelligence, and d) scalability.

On hardware, it's very expensive to purchase anything which can provide a fraction of the performance of a subscription. Traditional accounting depreciation would imply that purchasing local hardware is a terrible financial decision.

On electricity, this is a surprising cost center depending on location. A system with just one 5090 can easily pull 1kW, and to achieve usable performance for a workplace is going to require dozens of machines. This can represent an extra $10-20k in electricity in cheap places. In California or Europe this could be $30-60k per year.

As for intelligence, the frontier models from OpenAI and Anthropic are still superior, and they have at least a 3-6 month head start. Distilled models are closing the gap on some metrics, but they still can't compete. That's why they cost so much less.

The last major moat is the ability for subscriptions to scale with need. This means easily adding and removing licenses. This is far easier than purchasing extremely expensive hardware (and managing it), and selling it if/when internal demand changes. It's the same reason companies use contractors. The ramp up/down costs are very high.

The only real moat that local LLMs have right now is privacy.

This is exactly true, I get annoyed by Claude one day and switch to something else, and the only thing that's ever keeping me tied towards Claude is the ability to search my old chats easily.

But Claude also makes it really hard to do that, so what am I even really paying for? Time to extract all my data, put it into a sqlite with FTS5 and make sure I never rely on the overly-opinionated, low-thinking PMs from these giant orgs again.

Of course, that "easy" step has lots of partial solutions like CTK (Conversation Toolkit) or MyChatArchive and I haven't found the perfect one yet, ideally it'd be something that dumped everything into Obsidian or an Obsidian-alike, but surely somebody is working on that? I'd pay $5/month for somebody to solve that problem for me, as long as I still owned the data...

I knew that there was no real moat from the very start, I mean, these things were close enough from the very start, how could it not result in a race to the bottom, especially as you can't really prevent distillation reliably?

Same experience, all my other friends in the industry report the same; at work we went from a huge push for ChatGPT last year to switching to Claude, back to ChatGPT when it became cheaper than Claude, and in parallel a deployment of open models being trialed with mechanisms to route to other models when needed (and based on pricing).

Over time I can imagine us becoming mostly open models on our deployments when hardware is more accessible and the need for expensive frontier models is constrained to very few use-cases that might demand their capabilities.

Maybe open AI and Anthropic could just license their models to run on your own hardware. So a fixed cost instead of per token pricing or subscription with limits

They have good friends in the big ballroom to not allow you to use something cheaper and be locked in on them for your own safety

There are most definitely is a moat - but it works both ways. The railguards in the models create moats keeping customers out. And the cost to build a modern agentic model is in the 10 figure range and growing. This is an expensive arms race that is going to create moats.

I know a few Australian devs who work at places also moving, or already have, from Anthropic… and not because of cost but because of Trump’s edicts to ban non-nationals using AI.

Think about how crazy it is for non-US companies to use American AI providers - their marketing boasts that you can treat their models as co-workers, assign tasks, invite them to slack annd video calls, etc. Taken at face value, would you hire someone remote who lived in a country that commonly does random shit like deciding whether or not remote workers aren’t allowed to go to work?

The schadenfreude is that, at least for OpenAI, they were originally set up to make open models.

They were set up as a public benefit company and their returns were capped at 100x. They (well, Sam) went out of their way to put themselves in this death march to the IPO. If they had just done what Mark Zuckerberg did with Muse, they're not in this position.

Do you know how bad you have to be at the tech business to make Mark Zuckerberg look like a prudent-yet-visionary leader?

[dead]

I know what happens in many big companies and not a single one is moving away from Anthropic/OpenAI/SpaceXAI.

> Unless they both dramatically slash prices then they’re in big trouble

False, they have already done so many times.

> Neither of them can afford to do that and both desperately need to convince the street that the opposite will happen if they want any hope at a successful IPO.

False, margins are higher and I can have a formal bet that prices will go lower.

> However the cold reality for both is that there is zero moat to a model anymore

False, LLMs are not fungible and there exists a natural moat. I like the behaviour of Fable, not the behaviour of Opus - the fact that many people speak about this is evidence.

If companies are really doing this, then we're saying they have no problems spending tens of millions to get somewhat decent TPS and then having their employees complain they are timesliced and getting lots of timeouts because their org has 500 employees?

The open models are good because of distillation, which the US labs are actively working against via not revealing CoT ever and now you can see with OpenAI Astra 6 not even having a lot of CoT equivalents being emitted as tokens. Once the anti-distillation stuff is in place the open distillation models will probably start having larger and larger gaps.

If the companies survive the next few years, which they probably will because they represent too much of US economic growth to allow them to fail, this gap will keep on expanding.

Starting from zero without distillation is a lot harder, a lot more expensive and a lot more work. OSS models is what a laggard does to get adoption. China's gov't might keep on sponsoring it as a counter GPU embargo thing, but when gov't get involved, usually the other side gets involved too.

As for people asking where is the evidence for half of this, you will never have public evidence for most of this, but deduce what the partly hidden parts reveal about the whole and it is fairly obvious, especially if you look at the past behaviors of the governments and other actors.