As a decades-long Xfce user, I greatly value Xfce's modularity, and don't care the slightest bit about improving the display server performance. Xfce is already snappy well beyond my level of sensitivity, and I won't trade the flexibility I have and use for a sliver of extra performance I don't even think I might need.

(Yes, it's plenty snappy on an external 4K@60 monitor. A desktop environment is not a competitive FPS where a single extra frame of latency lowers your chance of being productive.)