alt Hacker NewsThere isn't much evidence at all that inference is "subsidised" (and by whom?)
Training is quite expensive and it does look likely that the American providers have been doing that at a loss.
In any case, you can go buy a MacBook Pro M5 48GB or an AMD R9700 and run Qwen 3.6 35B-A3B (a very capable model) and the only "subsidy" is you plugging it in, and 140W is not exactly a huge amount of power (roughly 50¢ per day if you run it 24/7 at 100% load, which it is very unlikely you will).
Replies
I agree with using smaller models, it's just that the majority of people I know feel like they need the biggest, beefier, behemoth model possible (with the longest thought setting) and consume much more than necessary when a flash or smaller model would be OK. I would also like to be able to use a smaller model, but given ram prices I would have to sell a kidney to buy ram now
> There isn't much evidence at all that inference is "subsidised" (and by whom?)
Well... why else would the major providers now tighten the screws on per-token pricing?
> There isn't much evidence at all that inference is "subsidised" (and by whom?)
None of the big providers are profitable. It’s subsidised by overly enthusiastic VCs.
> In any case, you can go buy a MacBook Pro M5 48GB or an AMD R9700 and run Qwen 3.6 35B-A3B (a very capable model) and the only "subsidy" is you plugging it in
Right, people could. But they won’t, because that’s a bloody expensive computer and they don’t need that to ask ChatGPT. That war is lost already.
Subscription to the big players’ services would need to increase massively for that to happen. And the computational cost is only part of the problem; these models also eat a lot of storage and RAM, which is not exactly getting cheaper.