Right, so we can distinguish between "model did an oopsie" and "model was actually hijacked".

The former seems to be a much bigger problem. I've seen lots of reports of "Claude/codex bypassed security / stole API keys to complete a task", where that was the "intended" behavior — at least from the LLM's POV, trained to persist and complete user's task at all costs.*

I've actually seen zero reports of actual hijacking due to prompt injection, though they must be out there too.

---

* See also the recent thing about how every model was willing to do fraud to get a refund:

https://news.ycombinator.com/item?id=50140462

They support payments through Shop and Link. I don't know about the former, but Link requires that you do a separate approval for purchases. The Duet cannot just take a card and do what it wants.

Sure. And I told Claude Code the other week to commit, merge and push some code.

It said "Done, you can open the local dev instance and see the changes".

I could not.

So I asked it. "Well, no, you're not able to see those changes because they're not on that branch or committed."

"But I told you to do that and you said you had."

Claude's verbatim reply?

"You're right. You asked me to do that and I said I would, then told you I had done it, but I had not. Would you like me to do it now, then?"