logoalt Hacker News

the_duke • today at 3:43 AM • 1 reply • view on HN

To be fair, a lot of the popular benchmarks have become pretty meaningless.

Models often get great scores despite being a lot worse than the frontier in practice.


Replies

anon373839 • today at 5:09 AM

This is true, but if you look at the change in weighting to specific benchmarks from one version of AA to the next, it looks really arbitrary.