alt Hacker News> but you can achieve the same at 1/10th of the cost.
For some tasks, sure. But not for all tasks. And for some tasks, cost per token is irrelevant if it provides real benefits that are oom compared to what you had.
Local models are indeed becoming "good enough" for some tasks, but there are still tasks that they can't touch. There's a recent benchmark for kernel writing. Fable wrote a kernel that provides ~30% more throughput per unit of compute compared to the latest Opus max / gpt max. Does it matter how much that session cost in terms of one session if you can take that kernel, deploy it on your inference fleet and "magically" get 30% more tokens served to your clients? There are companies that would pay millions for such a "leap". Because they can make more millions down the line.
Replies
The question is: what proportion of tasks can not be handled by GLM5.2?
How many software developers were working on code like the one you describe?
That’s true, there will always be demand for ultra-intelligent assistants, especially if they surpass what humans can achieve at similar cost. For the other 90%, the average frontier model will be good enough.
You're looking at the status quo and ignoring the trajectory. The best current open models are about as good as closed models from ~1.5 generations ago. The rate of improvement of all models is converging to zero. It follows that in a few generations, open models inferencing will be about as good as closed model inferencing.
The problem is going to become that there's no incentive for anyone to run the stupidly-expensive training phase. May God have mercy on the stock market.