This is the right idea IMO but I don't want to trust a third party tool for it either (no offense OP!).

Ever since I started running coding agents with shell access I've jailed them inside of VMs. Since I run Linux I can just use incus[0] for the VM management layer.

It's extremely simple, and you can vibe code a shell script to customize your workflow in a few minutes.

[0] https://linuxcontainers.org/incus/