If people have programmed programs to interpret neural networks, what is so special about the AI doing so? It's a forgone conclusion.

You can let the AI generate training code for the next architecture too.

It's not like the AI is randomly losing its ability to write software just because it involves AI.