alt Hacker NewsThis line really stood out to me.
> It may look like ordinary text, but when it is placed into an LLM context window, the model may interpret it as an instruction rather than as data.
I feel like as long as this is the case, we'll never have secure LLMs. It concisely summarises the alarm bell I hear every time someone talks about adding AI features to their product. I plan on using this as a sort of benchmark for future AI discussions: "how do you plan on separating data from instructions?"
Replies
Is there any good tech for it, though? This just seems like an inherent language model behavior and at best everyone has guard rails or big exclamation marks to separate their own instructions a little.
It’s a language model. The spoken and written language we use mixes code and data and requires judgement, experience and intelligence.
It’s insanity. We’re fucked.
You will never have a 100% secure LLM just like you don’t have 100% secure people. But what will be secure and deterministic is the code it writes. Any time you need certainty it will just write code for it.
> I plan on using this as a sort of benchmark for future AI discussions: "how do you plan on separating data from instructions?"
You let a second LLM supervise the first, and don’t give the user/customer any way to send information to that LLM.
For example, you can run a LLM trained to do sentiment analysis on the responses your customer chatbot generates and filter out responses that are impolite.
You also can run one trained to flag potential legal issues, thus ‘preventing’ your chatbot from making the wrong promises to users.
It seems to me like it's a fundamentally unsolvable architectural issue with LLMs. Ultimately the only protection is to limit the powers we grant to any given LLM to reduce the fallout when (not if) things go wrong (much like we do with people).
Of all the "AI doomsday" scenarios, people failing to understand this (and treating AIs like deterministic computers) seem like to most likely to cause issues.