alt Hacker NewsTo bang on the same damn drum:
Open Weight models are 6 months to a year behind SOTA. If you were building a company a year ago based on what AI could do then, you can build a company today with models that run locally on a user's computer. Yes that may mean requiring your customers to buy Macbooks or desktops with Nvidia GPUs, but if your product actually improves productivity by any reasonable amount, that purchase cost is quickly made up for.
I'll argue that for anything short of full computer control or writing code, the latest Qwen model will do fine. Heck you can get a customer service voice chat bot running in 8GB of VRAM + a couple gigs more for the ASR and TTS engine, and it'll be more powerful than the hundreds of millions spent on chat bots that were powered by GPT 4.x.
This is like arguing the age of personal computing was over because there weren't enough mainframes for people to telnet into.
It misses the point. Yes deployment and management of personal PCs was a lot harder than dumb terminal + mainframe, but the future was obvious.
Replies
I've seen this claimed, but I'm not sure it's been true for my use cases? I should try a more involved analysis but so far open models seem much less even in their skills. I think this makes sense if a lot of them are built based on distillations of larger models. It seems likely that with task specific fine tuning this is true?
Buy new Macs from where? There is a shortage of RAM, SSD, GPUs, and the CPU shortage just started.
That’s nonsense. Local models don’t have any of the nuance in text responses. I find them more akin to GPT 3.5 than even 4.x
The mainframe/PC analogy is spot on. And the hardware floor keeps dropping - you can grab a mini PC with 32-64GB RAM for a few hundred bucks and run surprisingly capable quantized models locally. Something like https://terminalbytes.com/best-mini-pcs-for-home-lab-2025/ shows the kind of hardware that's now available at consumer prices. The "scarcity" framing only makes sense if you assume everyone needs frontier-tier models for everything.