alt Hacker NewsI would do the same but my issue is that the models are changing so fast, so I don't want to be left out of the next model cuz it only runs on an even newer GPU or something like that.
But maybe my limited understanding is thinking of this wrong.
Replies
I wouldn't worry about hardware.
I've run the latest local models over the last year, including the recent Qwen 3.6 30B A3B, on a 9yo GTX 1080 and 32G RAM I have lying around[0]. If I can do that I don't think hardware will be a problem for you in the near term. The only updates I've needed were to Llama.cpp when a new class of model was released.
[0]: In my case, I want to see how local models perform on limited hardware, sacrificing context size and intelligence compared to SOTA models, so I have to really limit my expectations.
Focus on what’s actually required for your workflows.
Anything beyond that is just hobby, or continued education.
> I would do the same but my issue is that the models are changing so fast, so I don't want to be left out of the next model cuz it only runs on an even newer GPU or something like that.
I think the same, and it's why i stopped caring about running llama/etc at home last year. That coupled with the models being dumb by comparison to SOTA really make me fine with waiting.
But in a year or two it's going to be difficult to resist at home, assuming the pace of improvement holds.