alt Hacker NewsThe measurement metric is in-game steps. Unlimited reasoning between steps is fine.
This makes sense to me. Most actions have some cost associated, and as another poster stated it's not interesting to let models brute-force a solution with millions of steps.
Same thing in this case. No Utility and just as arbitrary. None of the issues with the score change.
Models do not brute force solutions in that manner. If they did, we'd wait the lifetimes of several universes before we could expect a significant result.
Regardless, since there's a x5 step cuttof, 'brute forcing with millions of steps' was never on the table.