alt Hacker News> Models are not free. Downloading them is free. Running them is not.
Is this really different from traditional software? Downloading postgres is free. Running it is not. You either buy hardware and assume the costs of owning and running that, or you pay to run it in the cloud.
Replies
aurareturn • today at 3:55 AM
I think the point here is that it takes the same hardware to inference an open source model as OpenAI/Anthropic inference their models.
IE, a lower param OpenAI/Anthropic model can compete with a higher param open source model.
So even if you are an American company who downloaded Chinese models in hopes of saving in cost, you still have to beat OpenAI and Anthropic in $/task which is very tough to do over the long run.
➕ show 2 replies
But inference costs scale per task whereas platform services like postgres typically amortize across tasks. If you are selling tasks done by inference, then compute is part of your COGS and it scales per task.