logoalt Hacker News

woah • yesterday at 9:12 PM • 3 replies • view on HN

The heavy reliance on eval datasets showcased here and also evidenced by the frontier lab's high payments for such data cast doubt on the claims that AI will become self-improving soon


Replies

anon373839 • yesterday at 11:27 PM

The frontier labs want you to believe that their models are magically gaining intelligence that generalizes to new areas with each new release. But it’s an illusion: the models are post-trained on a HUGE array of hand-curated datasets for specific tasks, with good enough coverage that it feels general.

its_down_again • yesterday at 10:53 PM

Curious, where did you see the numbers on how much frontier labs are paying for eval data?

➕ show 1 reply
charcircuit • yesterday at 10:25 PM

>doubt on the claims that AI will become self-improving soon

It already happened. https://github.com/karpathy/autoresearch