alt Hacker NewsProcedural generation can be useful for finding new musical ideas. It's also essential in specific genres like ambient and experimental music, where the whole point is to break out of the traditional structures of rhythm and melody. Imagine using cellular automata or physics simulations to trigger notes, key changes, etc. Turing completeness means there are no limits on what you can generate. Some DAWs and VSTs give you a Turing complete environment, e.g. Bitwig's grid or Max/MSP. But for someone with a programming background those kinds of visual editors are less intuitive and less productive than writing code.
Of course, often creativity comes from limitations. I would agree that it's usually not desirable to go full procedural generation, especially when you want to wrangle something into the structure of a song. I think the best approach is a hybrid one, where procedural generation is used to generate certain ideas and sounds, and then those are brought into a more traditional DAW-like environment.
I've actually tried all of the approaches that you've mentioned over the years, and - for my needs - they're not that compelling at the end of the day.
Sure it might be cool to use cellular automata to generate rhythms, or pick notes from a diatonic scale, or modulate signals, but without a rhyme or reason or _very_ tight constraints the music - more often than not - ends up feeling unfocused and meandering.
These methods may be able to generate a bar or two of compelling material, but it's hard to write long musical "sentences" or "paragraphs" that have an arc and intention to them. Or where the individual voices are complementing and supporting one another as they drive towards a common effect.
A great deal of compelling music comes from riding the tightrope between repetition and surprising deviations from that scheme. This quality is (for now) very hard to formalize with rules or algorithms. It's a largely intuitive process and is a big part of being a compelling writer.
I think the most effective music comes from the composer having a clear idea of where they are going musically and then using the tools to supplement that vision. Not allowing them to generate and steer for you.
-----
As an aside, I watch a lot of Youtube tutorials in which electronic music producers create elaborate modulation sources or Max patches that generate rhythms and melodies for them. A recurring theme in many of these videos is an approach of "let's throw everything at the wall, generate a lot of unfocused material, and then winnow it down and edit it into something cool!" This feels fundamentally backwards to me. I understand why it's exciting and cool when you're starting out, but I think the best music still comes from having a strong grasp of the musical fundamentals, a big imagination, and the technical ability to render it with your tools and instruments.
----
To your final point, I think the best example of this hybrid generative approach you're describing are Autechre. They're really out on the cutting edge and carving their own path. Their music is probably quite alienating because it largely forsakes melody and harmony. Instead it's all rhythm and timbre. I think they're a positive example of what generative music could be. They're controlling parameters on the macro level. They're not dictating every note. Instead they appear to be wrangling and modulating probabilities in a very active way. It's exciting stuff.