alt Hacker NewsThe original article already attempted to do that but I'll take another stab. Humans tend to believe what they want to believe. They gain confidence when a confident person (or LLM) they work with exudes confidence. Where they might have hesitated or got stuck in analysis paralysis alone, the aid of a con (short for confidence) artist will get them moving. They then attribute the solution and praise for it solely to the confident person (or LLM) that nudged them along. Did the "psychic" who understands human motivation better than the cops actually help? Very likely. Did the psychic end up getting far more credit than they deserve (it's magical! You'll never believe! Well worth the money I spent!)? Absolutely.
The amount of real help vs. confidence/manipulation the LLM or con artist provides varies in each situation and scenario and mix of people (and LLMs) involved. It's something to strive to be aware and analytical about.
There is also something else horizontally to this that occurs.
There are numerous scientific discoveries that have been made because one human looked at all available data and turned off the assumptions other people in the field have been using for years, maybe hundreds of years. Once you delete the assumption and make a new one the answer is obvious any anyone from that point wonders how so many humans could have missed it for so much time.
This can occur readily with LLMs as it can with people. It's very likely we will see this a lot as the causal connection in the available data will connected differently in their minds.
Escaping from a local maximum can be very difficult as you have to climb uphill with a nearly infinite amount of freedom but no ability to see the horizon. Psychics, LLMs, or some guy name Bob walking in an providing a workable analogy for you to escape the local maxima and move closer to a global maxima are all the same. Moreso, if LLMs are AGI we should expect these "psychic" behaviors just as much as actual discovery because both continuums exist in the same problem space as human minds.