Haven't people demonstrated all kinds of weak LLMs getting good ARC-AGI-3 scores with special harnesses?
Those people haven't verified their results against the private set: https://arcprize.org/leaderboard
alt Hacker News
Those people haven't verified their results against the private set: https://arcprize.org/leaderboard