I don't think this is actually true (eg. DPDK), but even if it is, you can put the driver in userspace (tun/tap + vfio/libusb/ioport/...) and still use TCP/IP in the kernel.