No. Absolutely not. The government owes its people a certain duty of care to say “just because you can doesn’t mean you should.”

LLM’s are good for advice 95% of the time, and soon that’ll be 99%. But it is not the job of OpenAI or any LLM creator to determine the rules of what good healthcare looks like.

It is the job of the government.

We have certification rules in place for a reason. And until we can figure out how to independently certify these quasi-counselor robots to some degree of safety, it’s absolutely out of the question to release this on the populace.

We may as well say “actually, counseling degrees are meaningless. Anyone can charge as a therapist. And if they verifiably recommend a path of self-harm, they should not be held responsible.”