If it's wrong 2 out of 5 times, why even waste your time going to it in the first place? That's a massive failure rate.
Because way more than three out of five Google results are SEO garbage or sponsored crap. The bar has been set extremely low by Google, a 60% validity rate sounds magical.
If I'm going to an LLM (as with websearch before it), it's usually because I don't know the answer, don't have anyone close to me that knows the answer, and can't pay anyone (or don't know who to pay) for the answer. In other words, my failure rate without the LLM would be 100%.
I don't find it nearly that bad. If I really need factual information, it will generally go off and read the data from primary sources anyway. So unless it's really misunderstanding context, you're getting the data from the source.
With Google returning lists full of SEO spam, 2 out of 5 is quite good. If you know something better than that, I'd love to hear it.
Because being right 60% of the time with minimal work is still amazing, as long as one accounts for the failure rate correctly.
Say I want to look up some game from my childhood, which I barely remember any details for. Going to google and trying is likely going to be very difficult unless I happen to get lucky with some key element. But if an LLM can get it right even a minority of the time, it can lead to me quickly finding the game I'm looking for.
This does depend upon the ability to evaluate the answer, like checking against source or some other option where you know a good answer from bad. If you can't, then it does become much more dangerous. Perhaps part of the reason AI seem to empower experts more than novices in some domains?
Because it finds the sources much quicker than I would have been able to on my own, and I can then synthesize them into data I know is correct, as correct as any human-generated data can be of course.