logoalt Hacker News

deepwoods • today at 1:34 AM • 1 reply • view on HN

The same evidence could just as possibly point to supply being too great for demand. The OP article references this as a distinct possibility: if models really have gotten more compute efficient, we may wind up in a situation where we have more inference capacity than we actually need.

To give a simplistic example: if a city as a ton of available AirBnBs, you might say, "Lots of people are willing to rent out their apartments, it must be profitable for them to do so." But the fact that all of those rooms are not already booked could also mean that way too many people have attempted to rent out their apartments.

And for what it's worth, it's possible for supply to outstrip demand while still leaving inference profitable. The question is whether it's (a) profitable enough to service massive debts and (b) sustainably profitable over the long run.

Edit: to add one note, it's possible for the LABS to be profitable on inference without inference itself being profitable. We don't know what the contracts look like, but it's entirely possible that labs received sweetheart deals from neoclouds so that the latter could flaunt big utilization numbers.


Replies

ThrowawayTestr • today at 2:14 AM

If inference wasn't profitable why are consumer GPUs that are only good for running open models so expensive? Why and how would these companies sell access to open models if it wasn't profitable? You can rent your own GPU and make money if your electricity is cheap enough.

➕ show 1 reply