logoalt Hacker News

vb-8448today at 3:36 PM1 replyview on HN

IMO in this case is mandatory to start from 0 because it alters the visual perception.

Just look at the first chart: the distance between Fable 5.1 and Sol is <5%, but it looks like 25 or 30%.


Replies

_aavaa_today at 5:18 PM

I disagree. The y-axis is some arbitrary intelligence score that we use as a proxy for performance on whatever our specific task happens to be. So it doesn't matter if a model is a 0, 1, or 20 along this axis, they are all useless for the tasks I want.

And as the complexity of your score increases, the cutoff goes up. We can quibble about where your personal cutoff is, but it aint 0.