> Surely if the machine you go to for answers regularly makes things up you would just stop using it?

Steve Yegge likened LLMs to slot machines. The human brain is very vulnerable to random reward systems. If you get an hallucination, just pull the lever once more.