Recently, I had a session where it said "I should do Y"

I said "No, do X. Research it. Y is based on wrong assumptions"

Thing did the research. Concluded with "Yes, indeed, Y was wrong"

Work is done

Turns out it did Y. But not just that. When asked about it, it not just (as LLMs do) made stories as to why, but "stood its ground" so to say.

"Refusing to perform regret", general snark and all that.

I know there is nothing really new or noteworthy in this comment or story, but I continue to be impressed by how misaligned the models of the company are, that can't stop talking about how incredibly important alignment is.

To the point where they now want to make it illegal (slice by slice) to push back against the clear and obvious misalignment of their models.

If you talk to people outside the industry about this, they just think you're telling them a made-up story. It's utterly absurd.