alt Hacker NewsAI pessimist view.
Even if AI gets smarter it will still be agreeable, and people will use it to more confidently reinforce thier stupider ideas, especially in areas where they lack the knowledge to know if they are right.
Even if this can be solved, technically, people won't want to use the model that says they are wrong, so they will choose the glib lies that reinforce their beliefs.
Freedown of speech forces people to think they may be wrong, but freedom of association lets people avoid this. AI is going to be internet hug box echo chambers at an unbelievable scale.
Replies
More recent AI tends to be less agreeable, sycophancy is widely recognized as a problem and AI companies are working to fix it. And sure enough, some people don't like it, as shown when OpenAI shut down GPT-4o.
But for the people who do actual work with AI, who are the people who pay a lot of money for it, they prefer to be called wrong when they are, because it is better to be called wrong by an AI than being wrong in front of a customer.
It is not an easy problem to fix, because as much as we want an AI to give us the right answer rather than just being agreeable, we still need them to follow orders, and AI that doesn't would be quite useless. So there is a balance to be found. Smarter models tend to do better because they know what is right, compared to smaller models that don't and then assume you are right and hallucinate from there.
Agree. AI behavior, when interacting with humans, is based on making money. Making people feel uncomfortable or challenged, will usually not be the preferred business strategy. But, it would be interesting if people could easily select the AI's personality type or to what extent it can be adversarial to the comments or ideas given as input.
Im not mad about this.
As someone who rarely uses LLMs because I dont need to, nor does it benefit me - I work on stuff that is original for which LLMs are useless at - Im glad.
I want everyone around me to get dumb as hell. It makes my path in life much easier and more successful.
Alas, a lot of people have always preferred a comfortable lie than an uncomfortable truth.
The solution is to stop treating AI as superhuman semi-living entities, as many boosters are apt to do. We can acknowledge it as a useful tool while understanding it has limits and is not “alive” or “thinking” in the same way a human is.
>. AI is going to be internet hug box echo chambers at an unbelievable scale.
More like hotbox of one's own farts, but yeah
Henry Petroski in "To Engineer is Human" has a chapter about the advent of the calculator during the slide rule era, and that calculators (and later computers) eroded a structural engineer's "feel for the numbers", caused engineers to pursue work on the fringe of their abilities, and perhaps more seriously, limited their ability to verify the correctness of the computer's work (this was the era where, for the first time, a machine could perform orders of magnitude calculations more quickly than humans could). He outlines the collapse of the Hartford Civic Center [1] as a case study in when blindly trusting computer output goes wrong (and by sheer luck, no one was killed).
It feels like the example was cherry-picked, but the combination of "less accurate" and "more confident" made me immediately think of this disaster. I truly worry that it's going to take civilian deaths / liability before the proper use / place of these new technologies is adopted, but we could probably start with 1) not in safety critical systems, and 2) don't replace your specialists / experts with algorithms.
[1] https://www.engr.psu.edu/ae/thesis/failures/MKP/failures/fai...