alt Hacker News How much does it cost to train a cutting edge LLM? Those costs need to be factored into the margin from inferencing.
How much does it cost to train a cutting edge LLM? Those costs need to be factored into the margin from inferencing.
Yes, which is why the companies that develop the models aren't cost viable. (Google and others who can subsidize it at a loss obviously are excepted)
Where is the return on the model development costs if anybody can host a roughly equivalent model for the same price and completely bypass the model development cost?
Your point is inline with the entire bear thesis on these companies.
For any use cases which are analytical/backend oriented, and don't scale 1:1 with number of users (of which there are a lot), you can already run a close to cutting edge model on a few thousand dollars of hardware. I do this at home already