"The LLM learns 2nd hand heresay, with no idea of what's true or false, what generalizations are valid, or what would be hallucinatory, " - do you know what is true and what is false? Take this: https://upload.wikimedia.org/wikipedia/commons/thumb/b/be/Ch... - Do you believe your eyes or do you believe the text about it?
I can experiment and verify, can't I ?