The problem is, that working on lots of little random code projects makes you fall into some kind of local minimum of overall joy and satisfaction. You are robbing yourself of the motivation of focusing on something truly substantial, that is still just a hobby, but the end result will leave you far more proud and fulfilled.
At the end of your life, if all you've done are little half baked throwaway projects, you might look back and realize one day you never made anything of any particular significance, just thrashed around building stuff people had already done so many times before that some unthinking, unfeeling LLM can spit it out almost verbatim just so you can say "me too".
This applies to more than just AI, it can be about any type of "side project" really, or any context where you have a wealth of so many possible options that focusing on one intensely forces you to deliberately ignore most of them.
An example for me lately is hackernews. I used to jump around wildy, looking at comments not really even reading articles. I felt like I was learning a lot. But lately I've taken another approach. Instead of clicking a bunch of things, I'm actually determining what is the most interesting article of the day, reading it thoroughly and truly thinking about it, and then after pausing for reflection, forming my own thoughts about it. I have found this to be a far more enriching experience than my previous habit. I think a lot of things in life turn out this way.
The only reason to use AI to build is when you don't really care too much about things, you just want something, anything. An image here, some code there, a ridiculous video. Cheap thrills with no soul required.
Idk, I'd put it in the same category as doing a crossword puzzle, building a LEGO set, doing some DIY task around the house, etc. A nice diversion that's not entirely creative but stimulating enough, and at the end you have something functional/interesting or at least satisfaction that a particular problem has been solved. It won't change the world or your life or make a million bucks, but not everything has to.