Basically whenever I use a machine that has an nvidia gpu, I always use xfce, as it just works, has least amount of issues & babysitting nvidia drivers & breakages. For everything else I use KDE.
I have some old chromebooks (flashed with chromebox firmware) that uses xfce too, which works great!
So kde & xfce is the only two desktops I use these days & have patience for.
Does the DE matter for your GPU? Can you give some examples of what xfce does better than kde when you've got Nvidia? Because I've got Nvidia and am using kde.