It is the imitation of intelligence.

It works because people have answered similar questions a million times on the internet and the LLMs are trained on it.

So it will work for a while. When the human generated stuff stops appearing online, then LLMs ll quickly fall in usefulness.

But that is enough time for the people who might think that it going to last for ever to make huge investments into it, and the AI companies to get away with the loot.

Actually it is the best kind of scam...

EDIT: Another thought. Thus it seems that AI companies actually have an incentive to hinder developements, because new things mean that their model is less useful. With the widespread dependence on AI, they might even get away with manipulating the population to stagnate.