One concern I have is API key management.

.env files or injecting secrets at startup via a secret manager still risks leaking keys.

I vaguely recall an implementation that substitutes secret placeholders with real secrets only during outgoing calls to approved domains which sounds better. However, you're still trusting an agent on your machine with command execution.