alt Hacker NewsBig number results, and impressive pricing. That said it really feels like benchmarks have been hyper saturated these days. I’ll wait for hands on before getting too hyped that Google is back. It would be nice having more than just OAI / A\ in the running for SOTA top tier intelligence.
Replies
xnx • today at 9:42 PM
~20% for Harvey's Legal Benchmark doesn't seem saturated.
nurettin • today at 8:21 PM
With these numbers, I'm holding my breath for the pelicanbench.
I don't think new benchmarks are saturated. They still give you a clue, they arn't perect but they have value. If model can't even do some easy tasks from benchmark then why would u even consider using it?