Is it considered a warmer color on the surface?
Mid-day sun in a clear sky is very white, in the 5k-6k color temperature range. It's hard to get a sense of how white it is because of how bright it is. In fact, the color temperature on the surface can be even higher than in outer-space!
Compare this to a "warm" light bulb, which is around 2.5K. Sunrise/sunset is also around that range.
Perhaps the "warm color" sun mindset comes from the only times that people can look directly at it. That is to say, around sunrise or sunset.