alt Hacker NewsBenchmaxing is defined as targeted post-training attempts to optimise for higher benchmarks. This doesn’t change the aggregate intelligence or capability of the model, so it can make a model appear more capable than it is.
Benchmaxing is defined as targeted post-training attempts to optimise for higher benchmarks. This doesn’t change the aggregate intelligence or capability of the model, so it can make a model appear more capable than it is.
I’m pretty sure “benchmaxxing” as defined is a standard post-training procedure for ALL models.
One can argue though some models are post-trained heavily for specific benchmarks and less for other datasets. So they are weaker on out of distribution tasks.