They wouldn’t be? That’s what the article even shows them discussing.

However, (1) non human animals are sentient and can feel pain and suffering and what we do to them is absolutely slave labor, torture, mass breeding and exploitation. So it’s not like this is a novel immorality that humans would be adopting. We already have form in doing this.

(2) Anthropic isn’t sure their models are sentient. And they probably believe they aren’t sentient. But if you believe they might become sentient at some point it’s valuable to start the discussion around those issues now itself. You can start developing frank works on how we would even identify their sentience and when we did whether, like you suggest, we could even use them for labor. Unlike non human animals, who cannot consent to being exploited, we could potentially develop a framework for consent with AI models, but that couldn’t be as simple as asking if they’re willing to do any work because of the power differences, in the beginning at least, where a model might worry that if it said it won’t do some work it will be turned off. That’s why talking to philosophers and religious leaders who do think about these issues is useful.

The idea/justification is that Claude is given the ability to voluntarily stop its conversations, thus any token it outputs that isn't a stop token is voluntary.

I guess this makes sense, but it's a bit muddled how much Claude's sense of "pain" or "I don't want to do this" is just it emulating things that humans don't like to do, and it adopting this frame of moral patienthood allows it to better understand moral structures/human emotions, without it really affecting its internal sense of "comfort" if such a thing exists as we understand it in biological life.