They are a bit weird with game development at the moment.

They can one shot entire games, with relatively minor issues.

And obviously asking for small code snippets and integrating them yourself has been well supported for five years.

But in Agent mode... not so much. I was asking frontier models to make simple changes to my Pong game (you know like the one from 1972) and it constantly failed to make simple changes or would break something else in the process.

The main issue is that they can't see what they're doing. Actually one of the agents tried playing the pong game by screenshotting every frame, and it ran for about 20 minutes before I realized what it was doing, and told it to calm down.

It takes about 10 seconds to process an image, so it was running the game at 0.1 frames per second... 600x slower than realtime. The technology is not quite there yet.

If your game is something turn-based though, with discrete States and well-defined transitions between them, they can help out a lot more with that.