logoalt Hacker News

BoorishBears • today at 7:33 PM • 1 reply • view on HN

Non deterministic models = "oh look they put up a lower score than Google did for that bench!"


Replies

underdeserver • today at 9:40 PM

I think the harness might matter more.

Assuming good intent they can run it internally a few times and post it with a disclaimer, but engineering a harness is a bit of a bigger ask.