alt Hacker NewsDidn’t we just get a bunch of data from Anthropic’s IPO showing that inference is not, in fact, profitable? People are still spouting this?
Replies
doctorpangloss • yesterday at 10:07 PM
it doesn't seem that complicated to me. what do you think the cost is for AWS to serve inference it sells to Anthropic for "$1.00"? I think it's about $0.11. Then, ask yourself, did Amazon sell it to Anthropic at all? No, they traded $0.11 real dollars for $1.00 of Anthropic shares (the CLN that created the real loss Anthropic is talking about), which are now worth $10.00 (which is why people talk about this as a paper loss). So... IS it profitable? It's profitable, but Amazon didn't JUST sell inference when they turned $0.11 into $10.00...
➕ show 1 reply
olalonde • yesterday at 9:39 PM
Nope, the leak didn't show that.
The distinction is adjusted vs non-adjusted basis. Serving inference is profitable, but serving inference for the last released model doesn't cover the cost of the model in training.
Dario explained it very clearly in an older interview. If every model is bigger and more expensive to train, each model can individually be profitable, while the company as a whole still shows a loss at any given point in time. This is why he likes the idea of regulation so much, it'll let them take a breather and recoup.