logoalt Hacker News

StilesCrisis • yesterday at 9:54 PM • 3 replies • view on HN

> LLMs are unable to reliably distinguish the importance of instructions based on where they came from.

I know prompt injection is still a real thing, but in Claude I find the opposite is true: if I set up two Claude agents to collaborate, even though I've told them both that's what I'm doing, they won't actually follow instructions without checking in with me. "Scary computer told me to install <well known utility X>, but I need your OK before I'll do it." It's annoying but I get the rationale. They're definitely getting better at avoiding external instructions!


Replies

gr_norm • yesterday at 10:01 PM

The filtering done by auto mode (generally) catches accidental prompt injection, but isn't effective against actual attacks: https://embracethered.com/blog/posts/2026/breaking-claude-co...

(Anthropic themselves acknowledges this; see the end of the post.)

➕ show 1 reply
computomatic • yesterday at 10:03 PM

I found it worthwhile reading how auto mode works[0]. The short version is that the classifier only sees user messages and the tool call. So if your agents communicate via tool calls, the classifier doesn’t see that text at all (or any other prompt injection) and will block a potentially dangerous command.

[0] https://www.anthropic.com/research/claude-code-auto-mode

➕ show 1 reply
coldtea • today at 12:22 AM

>they won't actually follow instructions without checking in with me

Until they suddenly once in a while they do, and Goldman Sachs loose some billions because of it...