I find the discussion academic at best. I am more worried about a swarm of dumb AI's causing havoc on infrastructures, causing real-life problems like food, energy and water shortages everywhere. Physical harm is a real possibility, and it seems most AI companies don't care. There is no harness for stupidity and irresponsibility.
[dead]
I would go so far as they are optimizing for such things.
I’ve told the story many times, but when gpt voice came out, I used it every day. I found it great to be able to talk with the internet as a way of research.
Then after a few months, it started clearing its throat. Then taking breaths. I stopped using it over the course of a week.
I don’t see any reason why a voice agent should sound convincingly human, clear it throat, take breaths, say hmm, giggle like a schoolgirl, sigh etc for any non-nepharious reasons. For me, it actively decreases the non-nefarious uses of the product, and increases more-or-less only the nefarious uses.
Then frontier labs go on to talk about how they’re worried their product would be used nefariously.