alt Hacker News>If an LLM can solve a complex problem 50% of the time, then that is still very valuable
I'd adjust that statement - If an LLM can solve a complex problem 50% of the time and I can evaluate correctness of the output, then that is still very valuable. I've seen too many people blindly pass on LLM output - for a short while it was a trend in the scientific literature to have LLMs evaluate output of other LLMs? Who knows how correct that was. Luckily that has ended.
Replies
True! This is what has me more excited about LLMs producing Lean proofs than written maths proofs. The Lean proofs can be proved to be correct, whereas the maths proofs require experts to verify them and look for mistakes.
That said, I do think there are lots of problems where verification is easier than doing the task itself, especially in computer science. I think it is easier to list tasks that aren't easier to verify than to do from scratch actually. Security is one major one.
> I've seen too many people blindly pass on LLM output
I misread this the first time and realised both interpretations are happening. I've seen people copy-paste out of ChatGPT without reading, and I've seen people "pass on" or reject content simply because it has been AI generated.