> "verify it rigorously (which if you cannot do, you should absolutely never touch an LLM!)"
100% this.
You have an automation bias. "Surely this thing knows more than me it must be right." and there is no reason to believe that, but you will.
Totally. And yet rigorous proof is very difficult. Having done some mathematics involving nontrivial proofs, I respect even more how difficult rigor is.
I like writing code more than reading it, personally.