I have never noticed that problem. But I always adjust brightness as part of a full color calibration, with ICC profile and everything, which probably makes a difference.

I will say that it took a bit to adjust to the first time I did it, but now it's painful to use any monitor at "full" 200+ nit brightness.