alt Hacker NewsYou can write your code so SQL injections are not possible.
You can't do the same with prompt injections.
Replies
This is true as long as “your code” includes the entire stack. There are still high level business applications where users enter SQL directly and it is only escaped, not handled using proper database SDK affordances.
LLMs are a decade or two behind SQL, but then they’re younger too. Just like we’re getting reasonable effected enforcement of output schemas, I expect we’ll see proper separation of control and data in the near-ish future.
It likely requires reworking model architecture since that’s single-stream now, but I don’t think it’s insurmountable.
Of course prompt injection will be a PITA for ages, just like SQL injection still rears its head today.
You have to have fixed commands that LLM could execute, just limit its universe. I don't think it is a good practice to give LLMs access to everything.
You can just make the tool calls restricted/scoped to whatever the calling account has access to (or in this case the repo)
That way even if the LLM broke out of the system prompt the worst case would be similar to a 404 or 401.
Why are we giving these processes super user access? No reason to have the executing loop/chat turns/tool calls be scoped to anything but the narrowest permissions.
If the agent truly needs data/permutations across different accounts or repos, treat the tool calls like any other API that needs to do that kind of work pre-LLM
You could just not have a user-facing AI agent
If you expose your private database's raw SQL access to public web, i bet people will find a way.
The same way here, i see the main issue isn't prompt injection, it is publicly accessible agent having access to private repos. What is the important use case for such a config that it warrants such basic security violation?
This. It’s unsolvable by design.