> they're trained a on a fixed set can only reproduce noise from here and there

This anti-AI argument doesn't make sense, it's like saying it's impossible to reinvent multiplication based on reading a times table. You can create new things via generalization or in-context learning (references).

In practice many image generation models aren't that powerful, but Gemini's is.

If someone created one that output multi-layer images/PSDs, which is certainly doable, it could be much more usable.

I've seen plenty of artists start by painting over an image they got from google image search, and end with something incredible.

And it's not that limiting. You aren't stuck with anything you start with. You can keep painting.