To be fair, a lot of the popular benchmarks have become pretty meaningless.
Models often get great scores despite being a lot worse than the frontier in practice.
This is true, but if you look at the change in weighting to specific benchmarks from one version of AA to the next, it looks really arbitrary.
This is true, but if you look at the change in weighting to specific benchmarks from one version of AA to the next, it looks really arbitrary.