This had real consequences during the HuggingFace attack- closed models refused to help, GLM-5 is what HuggingFace used to thwart the swarm.
More importantly, it showcases how little teeth the Claude constitution has. Claude will reassure users that its constitution means it prioritizes user autonomy and privacy, but will quickly admit how meaningless that is when "safeguards" override Claude.
That is to say, the constitutional approach to Claude seems weak and even misleading if said constitution puts no constraint on Anthropic.
Of course any approach that hands an LLM text to keep in it's context and hopes it will follow instructions is doomed to fail eventually. It's a probabilistic model, it's right there in the design of the thing that it will break the rules with some probability.
They even acknowledge that in the preamble:
> Training models is a difficult task, and Claude’s behavior might not always reflect the constitution’s ideals.
One need only look at their leadership and orbiters/relationships of said leadership to know this would always be the case.
> closed models refused to help
Yeah, I've always said that AI isn't AI if it needs safeguards that are so extreme it cannot figure out if something is genuinely bad or not.
And you believe this nonsense, that you needed teh help of a shitty LLM model and not a real expert on the topic?
I'm not concerned about alignment between Anthropic and Claude, I'm concerned about alignment between myself and Anthropic. As we all should be.