alt Hacker NewsI believe that, on a fundamental level, the principle of 'trust, but verify' can be followed to its logical endpoint, as covered in Ken Thompson's lecture, 'Reflections on Trusting Trust' [1]. At some point, one simply has to trust that something is correct, unless they have the capability to verify every step of a long chain of indirection.
So, in regard to your book: Claude may or may not have hallucinated the information from its cited sources. Gemini, as well. However, say you had access to the cited information behind a paywall. How would you go about verifying the information cited in those sources was correct?
Since the release of LLMs over the past four years or so, I have noticed a trend where people are (rightfully) hesitant to trust the output of LLMs. But if the knowledge is in a book or comes from another other man-made source, it's some how infallible? Such thinking reminds me of my primary schooling days. Teachers would not let us use Wikipedia as a source because, "Anyone can edit anything." Though, it's as one cannot write anything they want in a book -- be it true or false?
How many scientific researchers have p-hacked their research, falsified data, or used other methods of deceit? I do not believe it's a truly an issue on a grand scale nor does it make vast amounts of science illegitimate. When caught, the punishments are usually handled in a serious manner, but no telling how much falsified research was never caught.
I do believe any and all information provided by LLMs should be verified and not blindly trusted, however, I extend that same policy to works from my fellow humans. Of course, no one has the time to verify every single detail of every bit of information one comes across. Hence, at some point, we all must settle on trusting in trust. Knowledge that we cannot verify is not knowledge. It is faith.
[1] https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref...
Replies
This is great, your final line summarizes my thoughts as well. When it comes to matters of faith your average Redditor and Hacker News commenter will heap scorn and derision on religious people for accepting things blindly without any proof, yet they will blindly accept what other people tell them is true, or now what an LLM says is true.
> But if the knowledge is in a book or comes from another other man-made source, it's some how infallible
Nobody who's ever done research believes that. Everything gets put along a spectrum of trust/accuracy.
You should be able to say what you believe, what you base that belief on, what it would take to disprove that belief, and how likely you think it is to be disproven.
That's why you do research from as many primary sources as you can, because yeah, otherwise you're reading someone else's interpretation. Sometimes you can't do that (you don't read the language, etc) and then you have to judge the quality of the interpretation.
It's an enormous amount of work to write a book, and making things up doesn't make that process a whole lot easier. So most people try to be accurate. Especially with editors and such doublechecking work. I still always judge the quality of the work as I'm reading it.
LLMs just flat out can't be trusted. They're endless fountains of words and aren't accurate by nature. They're fine if you already know the answer, and not fine if you don't.