Yes, this really ought to be trained in (or at least RLHF'd in) but that would hurt engagement numbers so the opposite is done instead.

These are tools and it would behoove us all to keep that top of mind. Dangerous tools that are not your friend (but are useful as tools nonetheless)