logoalt Hacker News

torben-friis • yesterday at 7:16 PM • 3 replies • view on HN

I think the main problem is surfacing two topics:

- Understanding how effort and class interact (Is a high effort terra above a low effort Sol? Are larger models mostly always better regardless of effort?)

- understanding when (if ever) a new model tends to be better than an older but higher class one. Is an old astra better than a new terra?

If increasing effort and increasing model class produce improved results in a different way, I miss making it explicit in which way. In other words: What kind of issue warrants going up in effort? what kind of issue warrants going up in model class? What kind of issue is unlikely to be solved by either?


Replies

tedd4u • yesterday at 8:42 PM

OpenAI itself cannot answer these questions. A prospective price-sensitive (or speed-sensitive) user has to experiment to find the cheapest (or fastest) model that is sufficient for each task.

➕ show 1 reply
andai • yesterday at 8:17 PM

>effort / performance curves by model

Oh, easy:

https://deepswe.datacurve.ai/

Artificial Analysis has their own curves which are pretty similar.

➕ show 2 replies
dist-epoch • yesterday at 9:15 PM

> Is a high effort terra above a low effort Sol

You can literally see this in the charts from the article