It's better to think of QPUs as highly specialized coprocessors that do certain things really well. You would only every really use them for problems they're good at. Like ALU vs FPU vs GPU.