I had an interesting conversation with a guy at work past week. We were discussing some unimportant matter. The guy has a pretty high self esteem, and even if he was discussing, in his own words, “out of belief and guess” and I was telling him, I knew for a fact what I was talking about, I had a hard time because he wouldn’t accept what I was saying. At some point he left, and came back with “Gemini says I’m right! So, no more discussion” I asked what did he exactly asked. He: “I have a colleague who is arguing X, I’m sure is Y. Who is right?!”

Of course he was right! By a long shot. I asked gemini same thing but a very open ended question, and answered basically what I was saying.

LLM are pretty dangerous in confirming you own distorted view of the world.

This is probably right. In the past I've "blown people's minds" explaining what "the cloud" was. They had zero conception at all of what it meant, could not explain it, didn't have a clue. I mean, maybe that's not so surprising but they were amazed "It's just warehouses full of computers" and went on to tell me about other people they had explained it to (after learning it themselves) and how those people were also amazed.

I've talked with my family about LLMs and I think I've conveyed the "it's a box of numbers" but I might need to circle back. Just to set some baseline education, specifically to guard against this kind of "psychosis". Hopefully I would notice the signs well before it got to a dangerous point but, with LLMs you can go down that rabbit hole quickly it seems.

  > Nontechnical people simply don't have any idea about what LLMs are.

I think one of the big dangers is that people (including us) are quick to believe "I'm better than that". Yet this is a bias conmen have been exploiting for millennia.

The only real defense is not lulling yourself into a false sense of security. You're less vulnerable (not invincible) by knowing you too can be fooled

Honestly, it's just a good way to go about getting information. There's a famous Feynman quote about it too. The first principle is to not fool yourself, and you're the easiest person to fool

Let's be serious, it's not like AI companies haven't fed into this misunderstanding. CEOs of these companies love to muse about the possibility that an LLM is conscious.

"It would be astonishing if people were able to casually not antropomorphize LLMs"

Precisely. Even for technical people, I doubt its possible to totally disallow your own brain from ever, unconciously, treating the entity you're speaking to like a sentient being. Most technical people still will have some emotion in their prompts, say please or thank you, give qualitative feedback for no reason, express anger towards the model, etc.

Its just impossible to seperate our capacity for conversation from our sense that we're actually talking to "someone" (in the most vague sense).

> Nontechnical people simply don't have any idea about what LLMs are.

We need to be very very careful here. Just like advertisements work, weather you think you're immune or not, so does AI. You might think you're spotting every red flag, but of course you think so. You can't see all the ones you missed.

Do not make the mistake of thinking that being techy somehow immunizes you from flattery. It works on you too.

This is the best I’ve ever heard this put.