Better to put your agent on a zero trust private network, and force it to talk to a proxy with credential injection. That proxy doesn't need to have ingress, so your surface is basically prompt injections from files/web search and supply chain attacks.