Ah, no, that's not what I mean. It's the input devices. Mainly the mouse pointer.

I now remember there was a way to go around it (a bit cumbersome and ugly) which was to render the mouse pointer only locally. That means no mouse cursor changes for tooltips/resizing/different pointers in games, etc. But at least it gets rid of the lag.