As a heavy user of LLMs professionally, I too am so confused by the decision making process within these orgs. Do people really want them to speak like actual human connections? Why?
I doubt it, but it makes sycophantic managers feel good to see it serve up the same BS they spew.
I don't but RLHF probably does lead to this behaviour.