You might not always, just as we don't always keep humans in the loop when picking groceries for delivery[0]. Great. But that doesn't mean we can do that en masse.
You can give a workman a power tool and he shouldn't instantly assume that his housebuilding job is under threat, because corporations will build a house-building power tool.
Why won't he think that? Two reasons: 1) any tool that becomes more powerful tends to become more specialised and expensive to operate, so an electric drill doesn't scale to a house-building drill without incredible tradeoffs, and 2) he isn't reading a million breathless class warfare-igniting articles whose writers are paid through attention.
The comparison to worker with power tools is not great. Power tools are not replacing workers and it's absurd to think that drill would build house. If you wanted honest comparison you'd pick robots (and since we are talking about situation where AI is superior to humans in all knowledge work, then for that comparison to work robots would have to be superior to humans in all physical labour). Would robots displace that worker in this situation? Yes. Would it create new jobs? Yes, obviously, e.g. servicing these robots. But that's another job that could be done by other robot. So in knowledge work - if AI was better at finding and solving problems, better at making decisions and better at all aspects of intellectual work - what kind of new job would you see being created here that wouldn't be better performed by AI?
And as I've noted above that's jus assumption (to show that AI is different than previous displacements), I'm not making actual claim that LLMs will be better at knowledge work than humans.
> any tool that becomes more powerful tends to become more specialised and expensive to operate,
Isn't that exactly what is different about AI automation? AI is more generalized and cheaper to operate.