alt Hacker NewsMost people don't want to pay for inference hardware that sits unused 90% of the time. So concentrating that hardware in data centers does actually make sense.
Replies
wpietri • today at 5:40 PM
Is that always so? That was the theory behind the various video-games-in-the-cloud services, but I don't think that worked out.
➕ show 1 reply
zquzra • today at 5:42 PM
[dead]
> Most people don't want to pay for inference hardware that sits unused 90% of the time.
Enough people have gotten so badly burned by the "SAAS trap" that if people are actually making real use out of it, they'd be very likely to buy the hardware if it was at ~2024 prices.
This is the thing that Altman omitted from his «Local inference won't work out like ya'll think it will.» comment during that interview over beers a little while back.