logoalt Hacker News

adam_arthuryesterday at 9:55 PM1 replyview on HN

Storage doesn't require the same capex/upfront investment to get that margin.

How much does it cost to train a cutting edge LLM? Those costs need to be factored into the margin from inferencing.

Buying hard drives and slotting them in also has capex associated with it, but far less in total, I'd guess.


Replies

conradevyesterday at 11:22 PM

  How much does it cost to train a cutting edge LLM? Those costs need to be factored into the margin from inferencing.
They don't, though! I can buy hardware off of the shelf, host open source models on it, and then charge for inference:

https://parasail.io, https://www.baseten.co

show 1 reply