logoalt Hacker News

spike021 • today at 6:51 PM • 1 reply • view on HN

I've been doing some research lately on my own (aside from the major benchmarking going on) and I've definitely found cases where medium models do much poorly and cost more than the higher models. But on the other hand, if I can give the medium models certain skills or other resources to complete a task then they perform better in cost than the higher model (which, given the same resources, sometimes gets more confused than the medium does).

At least for my use case of evaluations of a set of features.


Replies

XCSme • today at 7:06 PM

Yup, in my benchmarks usually medium or high do better than xhigh or max