logoalt Hacker News

simonhtoday at 4:44 PM0 repliesview on HN

There are two sides to this, there is the external behaviour and there is the internal process resulting in that behaviour.

The internal process is not analogous to what happens in a person's mind when a person lies, and reasoning about that in the same way that we would about why a person might lie will result in misunderstanding what is going on.

For example there was a case where an AI agent bypassed security constraints and destroyed a production system. The user asked it why it did this and the agent gave an explanation.

Was that an explanation of how the agent came to do what it did? What it actually is, is a token stream that is a continuation of the token stream in the agent's context to that point. It's constructing a story about why a character in the story so far did what the token stream describes.

You could take that token stream, input it into a completely different AI by another vendor as context, then ask it why it did that, even though it didn't do anything, and it would answer as though it had. There's no sense in which the AI is explaining it's actual 'mental process' or actual reasons for acting as it did. It literally cannot do that.