alt Hacker Newsi don't know how more people don't experience this, but staring at an llms output be confidently wrong eventually to get there in the process, or looking at the commands its running and see some really stupid things its doing drives me crazy to the point i will just stop it before it answers, i cannot handle something being that confidently wrong (even though it'll probably still get there in the end)
Let's talk about being "confidently wrong." I don't know if the original author is reading comments here, but if so: how long has it been since you had the experience above?
Have you noticed it happens less and less often as time goes on? And that the models take less time and fewer trials to "get there in the end?"
Because the rate of change is the important part, not the state of the art at the particular moment in time when you formed your opinion.
Replies
No? LLMs have been outputting more text ever since chain of thought became a thing, and multiple researchers shown thinking traces are outright confabulations, only serving as more memory for latent processing. The tool calls too I believe have gotten more frequent since labs reward tool use regardless of practical value. The net effect is more total intermediate bullshit, even with more results.
OpenAI's harness can solve frontier math problems but the model alone is still a clofabulation machine. Its "intuition" did for a while get better from sheer scale, but we reached the optimum frontier for non-thinking models years ago.
I'd argue the improvement has come not so much from fixing hallucinations, but more in getting LLMs to successfully run for longer, which lets them more effectively fix their own hallucinations. This is still an improvement, but at the cost of bigger models, with more tokens, running for longer.
I'd be interested to see benchmarks for this sort of thing plotted against cost, power usage, and elapsed time. If these are all sub-linear then that's great, but if some aren't that could be a real problem for LLMs in the long run.