logoalt Hacker News

teiferer • today at 1:49 PM • 2 replies • view on HN

It's odd because I wonder if anybody actually cares about such scores. It's not like millions of end users look at the score and use it to make a decision what to buy. Subscriptions can easiliy be canceled and people anyway use the models they "feel" work best for them. Hardly anybody I know uses an even remotely scientific method to evaluate model performance. It's mostly vibes with a little bit of (mostly flawed) A/B testing sprinkled in.

So ... why bother to cheat?


Replies

InsideOutSanta • today at 1:54 PM

> So ... why bother to cheat?

I think part of it is to justify capex. "Yeah, we spent x billions on this, but look, we have the best models, we're beating OpenAI and Anthropic, we're winning, it's all worth it!"

dist-epoch • today at 1:58 PM

From all of the terrible benchmarks of AIs, the AA one is the least terrible.

➕ show 1 reply