logoalt Hacker News

jazzcomputerlast Monday at 10:59 PM1 replyview on HN

When you say "Things then get better and better with each model release."

How far behind are models that can be run locally, and do you expect that this will be widespread?


Replies

aspenmartinlast Tuesday at 2:35 PM

https://epoch.ai/data-insights/consumer-gpu-model-gap

I think over the years local models have fairly consistently been ~7 months behind frontier performance. Local models are hugely important but I don’t see the calculus changing. I can imagine it’s certainly the case for many tasks that there will be diminishing gains for performance improvements or reliability pass some threshold, in which case you don’t need frontier performance and you can certainly use local models or at least cheaper tiers of proprietary models if local is too much of a hassle. Plus of course use cases where local is necessary or the pros of having local models or on device models outweighs that of frontier.

Maybe things will change though, I would assume through basically government subsidies from China etc, to undercut existing frontier labs, but you can always spend more (better data more compute etc) for better performance and that I can imagine will always have a selling point.