alt Hacker Newsit looks like what they are doing is using a frontier model to write a simulator for a game and then solve using it.
it's not as impressive as it looks. the goals of Arc-AGI-like constructs is to get an IQ-like figure using raw'ish 2D measurement 'games' in the hope that it would signify something meaningful.
what this harness does is get the model to write a simulator first, it's measuring something entirely different.
Replies
Writing a simulator requires understanding the game well enough to spec the simulator. Acquiring said understanding - within the action limits of the benchmark – seems like the heart of the challenge, so this doesn't strike me as "cheating" at all.
This is classic goalpost movement. Arc-AGI-3 was launched this year with roughly 0.5% success for frontier models. being able to 99% it in less than six months sets a new record for Arc-AGI saturation timeline. Speaking of singularity measures. It is definitely a big deal, not least in that Chollet needs to cancel his summer vacation and write Arc-AGI-4 now.
The simulator the model builds is comparable to the mental model of the game humans create. It is also much more efficient, GPT 5.6 Sol cost $25,000 to run on ARC-AGI-3
> State grounding turns raw observations into objects, variables, and relations that can be tracked. Mechanism discovery finds how that state changes under an action and writes the rule as an executable program
The way I'm reading this isn't that they are writing a game simulator, but rather that they have two things they are evolving - a perceptual model of the game mapping from pixels to objects, and a behavioral model of how each action acts upon these perceptual objects. The behavioral model is written as a program that can be backtested by the game states and actions they have already taken to see if they are correctly predicting the resulting next game state.
The ARC AGI 3 games are non-trivial, and I think it's very impressive to see them doing well using this approach.
I'd agree with their conclusion:
> We read a saturated ARC‑3 as the new beginning: mechanism discovery as a general capability — grounding the causal structure of a world through the agentic loop of action and perception, in environments far richer than a 64×64 grid. This is where we are heading to.
This is the way that an animal learns about it's environment - by observation (and innate biases) to recognize the objects in the environment, and predict their behavior, both autonomous (which AGI ARC 3 doesn't test - the objects in the environment are passive), and in reaction to the animal's behavior. The animal predicts and observes, updating its predictions when it is wrong.
A system that could do this in a messy, dynamic, real-world environment would seem a like genuine step in the direction of animal intelligence, especially if it could ditch the symbolic representations.