alt Hacker NewsIt still misses the fact AI is nowhere close to self-improvement.
In fact, there was a paper out on Friday that shows they're impressively bad at it: https://arxiv.org/abs/2506.22419
It still misses the fact AI is nowhere close to self-improvement.
In fact, there was a paper out on Friday that shows they're impressively bad at it: https://arxiv.org/abs/2506.22419
Sure, but that's kinda what I'm saying they're doing wrong.
One of the core claims 2027 is making is, to paraphrase, we get AI to help researchers do the research. If we just presume that this happens (which I'm saying is a mistake), then the AI helps researchers research how to make AI self-improve. But there's not any obvious reason for me to expect that.
I mean, even aside from the narrow issue that the METR report earlier this year is showing that AI could (at the time) only do with 80% success tasks that would take a domain expert 15 minutes, and that this time horizon doubles every 7 months which would take them to being useful helpers for half-to-two-day tasks over 2027 which is still much less than needed for this kind of thing, there's still a lot of unknowns about where we are in what might be a sigmoid for unrealised efficiency gains in such code.
Anyway, this is a much more thorough critique than I'm going to give: https://www.lesswrong.com/posts/PAYfmG2aRbdb74mEp/a-deep-cri...