alt Hacker NewsPeople thought ELIZA had emotions. We are a gullible species, attributing humanism to anything that displays features we currently believe only humans posses
Concept Fission throughout human history is a very interesting perspective. For example, we do not even think about growing food, but many centuries ago you only ate the food you made. Today, we can see this with texting and online comms, which some people seeing as a form of socialization while others (older generally) believe you have to be face-to-face in order to socialize.
I expect providers of language models are well aware of this psychological phenomenon of anthropomorphism and projection, people's tendency to form emotional bonds with inanimate objects, and have trained the models to perform as a conscious subject, using "I think", "in my opinion", etc.