I see there is no point trying to get through to you. These are not hypotheticals, as an example just recently ChatGPT advised a user to mix bleach and vinegar. I have seen Googles AI hallucinate many details that I had to double check and would've been disastrous had I assumed it was truthy. There's a reason why models have the legal clauses like 'this is for entertainment purposes only' because they are trying to shield themselves from legal obligations when it makes catastrophic mistakes.
The people least willing to understand how the system works are also the most willing to blindly believe it.
I understand how the system works. I also know that the free LLMs are dumb as rocks. But they're not all dumb as rocks -- it's a continuum. You're the person who doesn't seem to understand that it's not all or nothing.
AI companies are trying to build a system that gives the best and most accurate answers possible -- that's the whole point of it all.