alt Hacker NewsWhat is the use of this arbitrary finding? There is nothing actionable from this only that I can think of. Happy to be enlightened here.
What is the use of this arbitrary finding? There is nothing actionable from this only that I can think of. Happy to be enlightened here.
Without having read the article, I would say it demonstrates that benchmarks alone, without knowledge of how a model was trained (c.q. constructed), cannot reliably determine whether or not a model has certain capabilities.