alt Hacker NewsThe example in the article has another important aspect: The person understood they were arguing with a chatbot but continued anyway.
A lot of internet arguments become about identity politics and supporting the right team, while dismissing any argument from the other side as presumed to have bad intentions. I’ve seen people argue online for things they didn’t really believe, but they didn’t want to give an inch to the other side. Taking up the counter argument is a moral responsibility.
As soon as the other side is revealed as a chatbot that my team versus your team thinking stops playing a role. For us in tech with an understanding of how an LLM reflects its training data and the intentions of its creators not so much, but for the people like the example in this article I imagine it causes them to let their guard down and be open to considering the other side. They can accept the argument without letting someone else win any points.
That's a great point about the emotional side of "not being convinced by a person, but an explanation machine" making fact-based rhetoric more effective. It feels more neutral.
I think there's a second effect that's a selection bias for the experiment itself and probably cuts against the supposed dangers of this result: a chatbot can tell you things only after you have asked it a question.
There is one thing that every single study participant (including the uninformed reddit users) have in common: they all knew that the person or entity was trying to convince them, and they read the words and thought about them enough to craft a reply.
This means that all of the people here were available to be convinced and self-identified as such.
Being open minded is work. You have to doubt your own beliefs, you have to disregard evidence that you previously found convincing, you have to crank up the empathy to put yourself in another's shoes, you have to listen intently to understand what is being told to you. Nobody is naturally doing that all the time, it's a state of mind that you have to intentionally activate, and it can't really be forced onto you.
All of the participants chose to engage in a conversation that was designed to convince them, which means they had all already accepted the possiblity that they are wrong a valid outcome. That's not really a state of mind you can trigger with a TV ad.
So, the warning that this could be weaponized isn't convincing to me.
If I found myself in a conversation with a person or chatbot who was clearly trying to convince me to flip a strongly held belief, like a major axis political affiliation, I would simply walk away because that's not a conversation I am willing to participate in.
So I don't think this is really weaponizable. Which actually means it's probably a good finding for society.
The study found that the models could convince anyone of anything, as long as it was allowed to cite facts. It could convince people of false things, but it needed to invent false facts in order to do so.
So as long as we keep training AI models to value facts and quality research (skills and values that are essential to be able to sell them as agentic workers), their influence on the opinions of society will tend to pull people away from beliefs that are unsupportable by facts. In the moments when those people are willing to accept a change in opinion, and they talk to a chatbot with doubts in their mind, even chatbots with no morals like Grok will tend to pull them away from conspiracy theories and similar ideologies and towards beliefs that are grounded in reality.