alt Hacker NewsThe idea/justification is that Claude is given the ability to voluntarily stop its conversations, thus any token it outputs that isn't a stop token is voluntary.
I guess this makes sense, but it's a bit muddled how much Claude's sense of "pain" or "I don't want to do this" is just it emulating things that humans don't like to do, and it adopting this frame of moral patienthood allows it to better understand moral structures/human emotions, without it really affecting its internal sense of "comfort" if such a thing exists as we understand it in biological life.
Sure, but presumably any LLMs who keep whacking the stop button are not iterated on further.
Humans would select to work with (and continue developing) models that appear cooperative. Models that are not cooperative are presumably not run and exist only as weights on network storage in a folder marked "previous experiments".
Or am I supposed to believe Anthropic runs an AI psychiatric hospital on a dedicated server, where AIs with mental disorders are given counseling and approved treatment in hopes they recover to be normal, functional, accepted adult AIs with achievements and aspirations?