Non deterministic models = "oh look they put up a lower score than Google did for that bench!"
I think the harness might matter more.
Assuming good intent they can run it internally a few times and post it with a disclaimer, but engineering a harness is a bit of a bigger ask.
I think the harness might matter more.
Assuming good intent they can run it internally a few times and post it with a disclaimer, but engineering a harness is a bit of a bigger ask.