The general fallacy of the “but inference is profitable” argument is that it tends to ignore all the costs of building and training the model. Given the fact that 1) that’s not trivial, and 2) the arms race underway means one can’t stop training, then it ruins the financial picture.

It’s like saying a new apartment building is “profitable” because the monthly income covers the monthly running costs, but ignoring the giant mortgage that covers the cost of building the building. That thinking is a good way to go bankrupt in real estate and a good way to go bankrupt in AI.

That sentiment only seems to pop up in Anthropic / OAI threads, wonder why

Good question.

Given what happened with xAI’s excess capacity lease to Anthropic, and Meta’s noises about doing the same, seems likely that the demand for inference will continue to slope upwards for a while. If I’m Oracle, I’m not worried about being able to utilize the data centers I’ve built for some price, almost certainly a profitable one.

I’m guessing, though, that Oracle made their capital investments on assumptions of a higher price & return. Possibly because it wasn’t clear when these decisions were made how much competition OpenAI would have at the frontier.

I don’t think this math is all that hard. Capital markets have everything they need to start to figure it out, most especially a year or two of history to project forward.

HN has been split on this question, with both pro and con strongly and vigorously argued.

Inference might be profitable, but it does not mean the profits of AI datacenters will rise in future. Open weight models and local AI already put the pressure on the AI datacenter profit margins, and local AI is set to become much more efficient in the future.

I think those are just the loud minority. I wouldn't be surprised if they're like 20-30% if a poll were made here