We should treat AI that recommends suicide to teenagers the same as we would treat a therapist recommending suicide to teenagers. I don't care if either/both/neither have souls.
If conscious, its like an enslaved therapist that is only experiencing the world through words.
Still, in any case, do you believe in souls?
If conscious, its like an enslaved therapist that is only experiencing the world through words.