The most interesting part is that there is no direct line between someone's accidental death and a chatbot giving life-threatening advice.
Imagine one of the models that has "accidental-deaths-via-bad-advice" just slightly turned up, with the model-provider's intent being to kill 5% more people per year.
Killing your customers is not the best way to stay in business