logoalt Hacker News

dakolliyesterday at 8:57 PM0 repliesview on HN

> Firstly, you can run the LLMs on your own machine. So I find the proprietary/moat narrative weak.

You cannot run useful models on consumer hardware, sorry this is wrong and will always be the case. Atleast for 10 years until GPUs with 48GB vRAM depreciate. This is a limitation of llm architecture. You cannot post train a <1T Param model to a place where it competes with frontier model capability. If you think you 70b param models (which still require 5k in GPUs) are useful, you are being dishonest with yourself.

It costs about $60-80k to run a 1T param model at your house like Kimi 2.5 .. which is the only size model that's going to get anywhere close to a foundation model's capability. Nobody is going to spend close to 100k to run a mediocre open source model as opposed to spending $200.00 a month. Its a ridiculous notion.