ADB uses libusb or WinUSB directly from userspace too yeah. On Windows you still need a .inf driver though because the WinUSB driver isn't getting loaded by default for Android devices because they're lacking the MS OS Descriptor