The answer to your question is that command-lines have a much larger diversity of syntax (even to get help!) than most people realize. Folks have their 30..60 commands they run frequently and don't run into many or conveniently forget/neglect older ones like `gcc` or `tar` or `dd`. Many people (not saying you specifically) do not even realize that double-dash long options are a GNU extension never standardized or that Python toolkits typically allow --my-opt for --my-option abbreviations, just to name a couple of the dozen variations (space or '=', or ':' or '/' or any of the above or etc., etc.). There are probably hundreds if not thousands of syntax possibilities, but people often act like there is only one.

As an example of diversity estimation that you can try at home, a couple of times I have run every single command in my command search PATH with --help </dev/null >/tmp/help.$c 2>&1 . Caution - be careful if you do this! Have backups/checksums of everything important and run as an unprivileged user. I always have to kill off several processes that just hang doing something or otherwise manually intervene. Anyway, this alone suggests data collection of help text is not a trivial problem.

Beyond data collection, many commands did not/do not use CLI toolkits at all. Their commands may have even less regular syntax. Freeform help makes it harder to produce a regular help syntax to convert into the interpreter needed by a completion system. That said, as elsethread commented for some toolkits the Zsh _gnu_generic works great! It essentially IS the "automagic" system you might want, just for a highly restricted circumstance.

Any CLI toolkit itself does have the data, by necessity. So, if the CLI framework supports the 2 or 3 common shells there is no need for a translator exactly. You just need a code generator. There is a stab at an auto-generation framework from said data for the Nim CLI toolkit, cligen, over at:

https://github.com/c-blake/cligen/blob/master/util/complgen....

but it only works for Zsh right now. Anyway, I don't think perfect should be the enemy of the good or anything like that, but you seemed to ask an earnest "why" question and these are some of the complexities.

People do!

See: https://pixi.carapace.sh/ or https://github.com/withfig/autocomplete

It's still a hard problem as lots of tools format --help differently. One of the things I'm jealous in Poweshell is their standardized completions