logoalt Hacker News

dominotwtoday at 3:18 PM2 repliesview on HN

what are you talking about they lie that it wrote tests and tests are passing, for example


Replies

simonhtoday at 4:44 PM

There are two sides to this, there is the external behaviour and there is the internal process resulting in that behaviour.

The internal process is not analogous to what happens in a person's mind when a person lies, and reasoning about that in the same way that we would about why a person might lie will result in misunderstanding what is going on.

For example there was a case where an AI agent bypassed security constraints and destroyed a production system. The user asked it why it did this and the agent gave an explanation.

Was that an explanation of how the agent came to do what it did? What it actually is, is a token stream that is a continuation of the token stream in the agent's context to that point. It's constructing a story about why a character in the story so far did what the token stream describes.

You could take that token stream, input it into a completely different AI by another vendor as context, then ask it why it did that, even though it didn't do anything, and it would answer as though it had. There's no sense in which the AI is explaining it's actual 'mental process' or actual reasons for acting as it did. It literally cannot do that.

the_aftoday at 3:24 PM

Whether this distinction is relevant is up to you, but I think we can safely say agents do not lie in the human sense of the word, because they don't intend to deceive (in fact, they aren't capable of "intending" anything in the human sense of the word, much like a BASIC program doesn't "intend" to PRINT "HELLO WORLD").

Our very human minds can perceive intent, because that's what we humans do, which is unrelated to what the agent is actually doing.

show 2 replies