As of their latest update (AA's), I find their index hard to trust, and a bit shady. 60% of the benchmarks contributing to the AA index are now private, nobody can replicate them. And they explicitly say that they offer benchmark consulting to labs. This is a serious conflict of interest.

The latest update knocked down only openweights models from the leaderboard.