Yes, this does happen. Solar farms have an albedo modification effect. The good news is that compared to fossil powered generation, the albedo modification is "paid back" in less than a year due to lower GHG emissions:

"Small reduction in land surface albedo due to solar panel expansion worldwide"

https://www.nature.com/articles/s43247-024-01619-w

If you download the supplementary information PDF, figure 19 shows a plot of GHG payback times by site. The most common time is about 1 month. The greatest time they found was about 9 months. These times go down as solar panel efficiencies continue to creep upward, because solar panels in 2026 don't have lower albedo than in 2024, but they do have better light-to-electricity conversion efficiency.