I think that is a bad argument.
You could ask a human to "do something with 1M dollars" and get garbage too.
This terrible argument comes up in relation to AI over and over again. I don’t care if you built a brain surgery robot that outperforms the average person at brain surgery, I care if you built a brain surgery robot that outperforms the average brain surgeon. Outperforming “a human” at a specialist task is still essentially worthless.
> You could ask a human to "do something with 1M dollars" and get garbage too
At least the human's brain requires about 20 Watts to operate.
Cheap garbage is better than garbage requiring whole power plants nad billions of capital investment.
The point is that the deciding is the "higher layer." It's not that LLMs cannot, but LLMs as designed do not. They are not designed for free-ranging days off, they are designed to help us do things, and that makes them somewhere between us and published works.
It's no different than an editor standing between the author and a book, or a publisher, or a printing press. It's part of the process of idea realization, so it's just on that spectrum somewhere between book sale and author idea. (even though in another flipflop, that editor might be an author and now elevated to the premier deciding role). It's about how the tool fits, not what the tool is ultimately capable of.
So the analogy holds, just fine.