alt Hacker NewsThis critic also makes fun of them because they go on and on and on about how vitally important it is to produce a safe tool that won't do harm, when their core products frequently consider attacker-controlled instructions to be its system instructions or its user's instructions, and are known to confuse their own internal chatter as instructions from their user.
Reliably differentiating between trusted, tainted, and untrusted data and ensuring that you don't mix the latter two groups in with the former is something we've known to do for nearly a half-century. Hell, even the youngest plausible programmer at the LLM companies is all but certain to be aware of SQL injections. And yet, despite their claims about being so serious about safety, they show zero interest in following long-proven software safety practice and rearchitecting their software to make it impossible to mix system, user, and attacker-controlled data. [0]
[0] One might argue that the fundamental nature of LLM-based systems makes this impossible. If that were true, then it would mean that these systems are impossible to make safe... the only safety option available would be to establish comprehensive blacklists, which is simply infeasible.
LLMs are impossible to make safe in the same sense that humans cannot be made safe. There is no such thing as out of band data in the human mind.
For example, you have a dictatorship and need to track what the democratic countries are up to. The vast majority of citizens don't have access to information so will remain indoctrinated, but how can you be sure your data analysts will remain that way? You can't. So you take a batch out and shoot them at regular intervals.
The only winning move is not to play, but we're already past that point.