alt Hacker NewsAccounting for initial acquisition cost and typical depreciation/failure rate? I assume they already consider colo price, electricity, etc
Replies
petesergeant • today at 11:33 AM
I would be surprised if there was such a glut of undifferentiated inference providers for open models otherwise[0]. Possibly just people trying to minimize the loss from already-committed capital, but I don't see AWS and DO selling inference via OpenRouter unless they're making a decent profit from it.
70%+ net profits margins are for rented infra, that means no major upfont expense. Also, these margins are against 60% utilization only.
You can well imagine if you own your infra and can make utilization over 90%, where these go.
And then, my link above is for K3 model, which, compared to ds flash models, is highly unoptimized.
If I had a few million lying around, I would have invested in inference.