If your threat model is that of a malicious agent that will use a 0-day LPE to get root and exfiltrate all of your SSH keys, virtualization makes sense. But then, I wouldn't run such an agent at all, if not specifically in the context of malware analysis.

If you're just concerned about "agent messing up and taking the rules in some markdown files more laxly than I would have", then running it as a seperate user is totally enough...