MentalHealthBench

Mental health plus ad profiles, what could go wrong? I can't help but wonder about the moral fiber of the mental health professionals they say they consulted for this.

This seems to miss the point for me, therapy is a series of 20+ conversations, steered by the therapist.

Current LLMs would instead be steered by previous tokens, meaning the patient does. So someone that for example has schizophrenia may actually convince the LLM it's true.

I find this troubling, there are so many potential ethical issues with this application of language models. Maybe it would be more appropriate for models to issue safety refusals and help the user learn how to get help from a licensed mental health professional