> The weird thing for me is the prompt optimization loop? Why not fine tune the model instead of AI generating the prompt?

Why is it weird to optimise the prompt? Whether you optimise the model is a separate issue.

If you use any closed models you can’t fine tune them, which is another reason for most but here they also fine tuned models.