logoalt Hacker News

simianwordstoday at 7:03 AM0 repliesview on HN

On one hand I agree with ARC team that harnesses can incorporate some logic and overfit to win the tests. But also, the specific harness that OpenAI wants to use is not cherry picked. What then should be the rules? Same harness for all or company’s standard commercial harness? Technically even that could incorporate some hacks.