logoalt Hacker News

BrokenCogs • today at 6:22 PM • 2 replies • view on HN

Every release I see a new benchmark with a new frontier. These graphs are just meaningless now


Replies

abeppu • today at 6:29 PM

In particular, where cost is the x axis, a well-funded company with a good but not best model can discount their new model until it reaches the frontier. In some sense this is as much a statement about the efficiency of markets in the presence of public measurement-based comparison and public list pricing as it is about research pace.

➕ show 1 reply
yipinwong • today at 6:29 PM

you can give it a meaning if you see the relavivity between models. no need to disregard the stat at all.