alt Hacker NewsSomeone above posted a link to wardgate, which hides api keys and can limit certain actions. Perhaps an extension of that would be some type of way to scope access with even more granularity.
Realistically though, these agents are going to need access to at least SOME of your data in order to work.
Author of Wardgate here:
Definitely something that can be looked into.
Wardgate is (deliberately) not part of the agent. This means separation, which is good and bad. In this case it would perhaps be hard to track, in a secure way, agent sessions. You would need to trust the agent to not cache sessions for cross use. Far sought right now, but agents get quiet creative already to solve their problem within the capabilities of their sandbox. ("I cannot delete this file, but I can use patch to make it empty", "I cannot send it via WhatsApp, so I've started a webserver on your server, which failed, do then I uploaded it to a public file upload site")