logoalt Hacker News

Alexwortegatoday at 5:10 PM1 replyview on HN

Do you think it will be possible to train on this bench?


Replies

matt4711today at 5:16 PM

The judgements are available on huggingface so training is possible. But given that we evaluate on new queries daily there would need to be some generalization happening for this to show up in the benchmark.