logoalt Hacker News

TeMPOraLyesterday at 8:51 PM1 replyview on HN

> Base64, Unicode substitution, emojis, output of obfuscated but "harmless" code run in a sandbox, image steganography, etc that could be endlessly disguised without a human even being able to see it, yet alone fall for it

Like a whisper or a morse code pattern or a post-it stuck in the middle of a stack of fresh printouts saying "${employee} is threatening to kill me please call 911" or...

Yes, LLMs and humans have different sensory inputs. That's immaterial; the "problem" isn't in the intersection of LLM and human sensoria, but in what happens once those inputs reach the model/brain.

> I always hear the "but humans fall for social engineering too!" line used reflexively whenever yet another prompt injection attack gets reported and it drives me crazy.

Weird, because all I see over this site is people reflexively claiming that prompt injection is like SQL injection and "lethal trifecta" is a software engineering problem.

It is not. Prompt injection is, fundamentally, the same class of thing as social engineering. Treating LLMs as regular software is a fundamental category error. Until people finally get that, we won't get much progress in this space, because all the effort gets wasted on barking up the wrong tree.


Replies

lovichyesterday at 10:39 PM

Agreed. I do not understand why so many others cannot understand that independent non deterministic models cannot be constrained the same way as deterministic code can.

It’s just fundamentally different. They are both software in the same way liquids and solids are both matter but they have fundamentally different properties due to their nature.