alt Hacker NewsThese AI 'relationship' type bots are everything wrong with tech.
>> Megan Garcia had no idea her teenage son Sewell, a "bright and beautiful boy", had started spending hours and hours obsessively talking to an online character on the Character.ai app in late spring 2023.
People become obsessed with them. The builders have to know that their 'customers' are explicitly people with mental issues. Nobody sane or normal is talking to these things.
If you want to see how bad it is go checkout the reddit discourse when OpenAI deprecated one of their older models. Thousands of people acting like OpenAI had 'killed' their partners and best friends.
There are a lot of grey areas engineers work in when it comes to social stuff, privacy stuff, etc. There's no grey area with these. You're trying to hook people who are unwell and the people working on it should be ashamed.
None of this pertains to the legal case whatsoever. If you think chatbots should be legislated out of existence, you are welcome to your opinion, but while they exist, trying to hold a particular company legally liable for a chatbot saying "come home to me" is beyond absurd.
---
Edit replying to below post, as I am rate limited:
> Talk about misrepresentation. Either way my comment didn't mention the legal case. I was simply pointing out that anyone working on building those types of bots is sick. They take advantage of vulnerable people, milk as much money as they can from them, and want to wash their hands of any responsibility when it eventually goes wrong.
You replied to my post, so I thought your post perhaps had some relevance to mine rather than being unrelated soapboxing.
I don't particularly agree with your soapboxing, at any rate. Character.AI was not a "relationship bot" company. Like any LLM, they could simply be prompted to respond as such, in the same way that ChatGPT can. As you pointed out yourself, ChatGPT has the same issue with people forming parasocial bonds, despite not attempting to cater to that market in any way at all. Should people who release chatbots be legally required to censor them heavily when users attempt to use them for anything other than technical questions? That seems excessive, and it seems that ascribing moral responsibility of that degree is akin to holding video game, music, or movie producers responsible for violence committed by someone who saw a piece of violent media. Moreover, how far does it go? Should distributing open-weight models be made illegal, because you're making available something that can't be censored?