I've tried to passthrough Behringer audio interface, as well as HX Stomp (to, of course, avoid dual-boot) and it stuttered like crazy. Maybe there is a way to make it more "realtime" but it's not the default, at least in VirtualBox.
VirtualBox is the worst VM software.
QEMU on Linux with kvm is gold.
Yeah passing through usb devices to a VM for audio seems to be a dark art like working with a bunch of analog equipment. Everything has to be tuned. I do understand why a bare metal windows and/or mac computer is always in a studio, one can skip the head ache. Even if main DAW is Linux.