alt Hacker NewsSecurity risks aside, that's pretty remarkable problem solving on Claude's part. Rather than hallucinating an answer or just giving up, it found a solution by creatively exercising its tools. This kind of stuff was absolute sci-fi a few years ago.
Replies
A sufficiently sophisticated agent, operating with defined goals and strategic planning, possesses the capacity to discover and circumvent established perimeters.
Honestly, I think many hallucinations are the LLM way of "moving forward". For example, the LLM will try something, not ask me to test (and it can't test it, itself) and then carry on to say "Oh, this shouldn't work, blabla, I should try this instead.
Now that LLMs can run commands themselves, they are able to test and react on feedback. But lacking that, they'll hallucinate things (ie: hallucinate tokens/API keys)
Or this behavior is just programmed, the old fashioned way.