logoalt Hacker News

woggyyesterday at 8:52 PM8 repliesview on HN

What's the chance of getting Opus 4.5-level models running locally in the future?


Replies

dragonwriteryesterday at 9:29 PM

So, there are two aspects of that:

(1) Opus 4.5-level models that have weights and inference code available, and

(2) Opus 4.5-level models whose resource demands are such that they will run adequately on the machines that the intended sense of “local” refers to.

(1) is probable in the relatively near future: open models trail frontier models, but not so much that that is likely to be far off.

(2) Depends on whether “local” is “in our on prem server room” or “on each worker’s laptop”. Both will probably eventually happen, but the laptop one may be pretty far off.

lifetimerubyistyesterday at 11:17 PM

Never because the AI companies are gonna buy up all the supply to make sure you can’t afford the hardware to do it.

SOLAR_FIELDSyesterday at 8:53 PM

Probably not too far off, but then you’ll probably still want the frontier model because it will be even better.

Unless we are hitting the maxima of what these things are capable of now of course. But there’s not really much indication that this is happening

show 3 replies
teejyesterday at 8:56 PM

Depends how many 3090s you have

show 1 reply
kgwgkyesterday at 9:29 PM

99.99% but then you will want Opus 42 or whatever.

rvzyesterday at 10:26 PM

Less than a decade.

greenavocadoyesterday at 8:58 PM

GLM 4.7 is already ahead when it comes to troubleshooting a complex but common open source library built on GLib/GObject. Opus tried but ended up thrashing whereas GLM 4.7 is a straight shooter. I wonder if training time model censorship is kneecapping Western models.

show 1 reply
heliumterayesterday at 9:29 PM

RAM and compute is sold out for the future, sorry. Maybe another timeline can work for you?