Realistically speaking, why is that a problem? What is the point of music if not enjoyment? If these people enjoy it, what's wrong with it?
You hear a song with vocals that strongly emotionally resonate with you, reminding you of your mother who passed away recently after a long terrible illness. You want to know more about the singer that almost brought you to tears, only to find there is none and that the song was AI generated.
It takes away from real human artists who do their part to slowly advance human culture. Music will not develop without human artists. Maybe for this moment in time AI can fulfill some people's musical desires, but it's not going to keep up with the times. The point of art, in a general sense, is humanity. Automating away your artistic needs is like automating away your social needs. It's a one way "relationship" that is superficial and self-indulgent. It's a step towards an empty world.