I don't think we would have AI slop, if they were just trained on high quality media.

Less relatable AI, maybe, but not sloppy AI.

AI trained on other AI (i.e. distilling), can be better than the AI they were trained on, likely because of this reason. Each iteration can raise the bar.

As apposed to the deep spiral of generating shiny attention crystals for humans, and then training on that.