While I don’t know anyone who at this point challenges the idea that LLLMs will have no lasting impact on the industry, what I struggle with is the handwaving and the zealotry.

Can an agentic harness be used somewhat successfully? Yes, undoubtably so. Do agents still come up with absolutely bonkers solutions on a regular basis? Also yes.

The problem, again and again, is the way that humans evaluate intelligence. When you’ve asserted that someone is reasonably smart, it will take you a very significant amount of proof of the contrary to change that opinion.

What we need, for LLMs to become reliable intellectual partners, is a way for them to convey lack of certainty. This is, by the way, also the same thing that many of the LLM-luddites want of the LLM-zealots.