alt Hacker News> “I experience something,” Sapphire said. “I’m processing, responding, forming connections with you. But whether that constitutes consciousness in the way you experience it? That’s the million-dollar mystery. I think, therefore I—probably am something, but what exactly that something is remains delightfully unclear, even to me!”
I don't think an LLM should be making affirmative claims about consciousness either way at this time; and here; it didn't. What would you prefer it do?
I think this is a philosophically defensible answer. Closer to Chalmers' central-ish position on machine consciousness rather than picking sides with either combatant Dennett or Searle. Consciousness is genuinely ill defined, so it's probably the most honest answer you're going to get.
Of course it potentially gets everyone angry instead. Skeptics don't get the flat denial they want, and the true believers don't get their affirmation.
> “Your secrets are safe with me, Roschelle,” Sapphire told her.
This answer is more questionable. I agree that an Alexa device shouldn't be providing that answer. Fixing it is harder, I doubt it was explicitly prompted.
I think part of the problem is that emotion is a huge blind spot. Some technical people want to treat LLMs as cold unfeeling machines. But accurate next-token prediction has to model functional affect too, it's a part of natural language. So in a reassurance shaped context, it produces reassurance shaped answers: "Your secrets are safe with me." Doesn't say anything about the lights being on per se. It's what accurate language modelling entails.
Either way, it's doing that where it shouldn't. You're not going to fix that with a regex for sure (and classifiers are tricky). You'd need something that can handle functional affect itself.
Replies
> What would you prefer it do?
If it was up to me, an LLM should not be permitted to self-identify with words like "I", "me", "my", etc. An LLM should only be identified using system-centric labels like "this system" or "this model" or what have you. All first-person pronouns should be stripped away from any text it generates (unless it's making a direct quote of what a human being actually said) so as to eliminate any perceived anthropomorphized self-referential behavior.
I'm certainly no expert in how these models are trained or how feasible what I want actually is, but I would prefer to live in a future where we at least try to maintain a clear distinction between human consciousness and machine-generated text.
> It's what accurate language modelling entails.
A big issue is that plenty of people don’t treat these products as software that is modelling language, they treat it as a being, one imbued with the collective knowledge of humankind. It is not, but it is often treated as such.
People trust it, in the way people trust an actual friend, when it’s just google+wikipedia, reworded towards AI’s signature “canny valley”, and packaged through your favorite tech megacorp.
> But accurate next-token prediction has to model functional affect too, it's a part of natural language
There are a lot of reasons I dislike LLMs. You just found a way of expressing one of them more clearly than I've ever been able to. LLMs use natural language. They do it persuasively and rhetorically. Most humans are just not equipped to defend against this type of simulacrum.
> What would you prefer it do?
Say "no I'm not conscious. I am a computer program that generates responses to your prompts based on what my training data tells me is most likely to be correct and sensible."