Fully agree that in the end sandboxes are required to get frontier performance out of the models.
But you can have both: rund the agent outside the vm/sandbox and orchestrate work on it, either directly via shell calls or kicking off an ephemeral subagent on the box.
This makes the agent and session that runs outside the vm more durable and opens new orchestration pattern.
I’m building the oss version of this here: https://github.com/smartcomputer-ai/lightspeed