Number of iterations solved correctly, on a very large set of iterations, seems like a very good metric. Better than anything else because it measures what we actually care about, not some proxy.

The only caveat is that it's the same model doing an iteration and then using that iteration as a starting point for the next step. So the model is allowed to write absolutely insane solutions, as long as it can read them back, even if no one else can.

One thing that could be done is to use a separately developed baseline coding model B to evaluate the outcome of each iteration. For model under test X to pass an iteration, not only should it be able to solve starting from the previous solution, but so should B, starting from X's previous solution.