alt Hacker NewsI would argue that these are products and companies want people to use them. Like it or not, a model that disagrees with some fundamental core of your beliefs will make you much less likely to use it. To me it's the same problem with LLMs just being overly agreeable. The general public doesn't want tools that argue with them.
Which leads to a problem. Training multiple models is hella expensive currently, so they have to attempt to make them 'neutral' as possible. Which is something that doesn't work when you're trying to sell a product globally. You'll end up with people and governments arguing it tell them the right answer when they ask about god.
I'm guessing at some point in the future we'll have a lot of different distillations of the same model caters to particular regions/beliefs.