I was picturing one-shot performance only for the benchmark, on novel real world tasks. I.e. the score on the March Olympiad you got in April isn't relevant.

Simple enough that anyone could run it with a regular subscription.

Really unless we can get the providers to ditch the gameable benchmarks they won't.

But industries love nothing more than a benchmark they can manipulate.