For local tasks you can only give agents delegated that execute your deterministic read or write on an allowed set of files(e.g pi does this) and execute rights only on containers with no network access. That should get you 95% unblocked for most tasks you want to do with an LLM pretty safely.

You can do a brainstorming with web on a remote container prototyping based on that brainstorm on another container with no network access.

The one thing that is less trustworthy is using local agents for service management, you definitely want to have them scoped to dev/testing. I would never trust an agent to execute any command in production or sensitive data at all

Is the trust concern for the agent running in any form on your machine? Like in a VM on your machine as well or do you mean on the host itself?

I have read about people giving an agent full access to their main system saying they have nothing of value. To me, that's a strange opinion to have with the distinction between what's private and what's secret.

Do you also run your browser in the VM? Why would an agent be less trusted than any other piece of software?

I mean, if the execution happen on the VM then the problem is trust on the programs and then you can't trust any program by that logic? That or you think AI-companies software is serious slop.