> Normally if LLMs want to compose multiple operations, they have the perfect tool for this: bash, or whatever other OS shell is available.
I many scenarios, e.g. running the harness server-side, as is the case for chat interfaces, you don't really want to expose OS shell access as that opens up a huge security attack surface.
You can give the LLM a bash without giving it the full /usr/bin.
That's been a trivially solved problem for decades.
> I many scenarios, e.g. running the harness server-side, as is the case for chat interfaces, you don't really want to expose OS shell access as that opens up a huge security attack surface.
It does, but a restricted user account mitigates the large majority of those issues. A sandbox mitigates even more.
The number of remaining exploits left is probably going to be the same as the number in the harness. More, in fact, as many of them have no human review anyway.