logoalt Hacker News

Hasslequestyesterday at 3:54 PM1 replyview on HN

Firstly, you can run the LLMs on your own machine. So I find the proprietary/moat narrative weak.

Secondly, I find that correct usage of LLMs can accelerate learning. My brother used an LLM to generate flash cards for a driver's license test. I use LLMs to digest a ton of text and debug issues that would have been impossible to find (I would have given up) Have it generate, explain, review, compare code or general writing.

It is like having access to wise old man in every field. They may have inferior reasoning capability, and their memory may falter, but they have seen everything in their corpus and are great at pointing you to external references. And you can delegate them to busywork.


Replies

dakolliyesterday at 8:57 PM

> Firstly, you can run the LLMs on your own machine. So I find the proprietary/moat narrative weak.

You cannot run useful models on consumer hardware, sorry this is wrong and will always be the case. Atleast for 10 years until GPUs with 48GB vRAM depreciate. This is a limitation of llm architecture. You cannot post train a <1T Param model to a place where it competes with frontier model capability. If you think you 70b param models (which still require 5k in GPUs) are useful, you are being dishonest with yourself.

It costs about $60-80k to run a 1T param model at your house like Kimi 2.5 .. which is the only size model that's going to get anywhere close to a foundation model's capability. Nobody is going to spend close to 100k to run a mediocre open source model as opposed to spending $200.00 a month. Its a ridiculous notion.