> If we were trying to adjust the time to track the solar time, wouldn't we need to adjust the clocks every day as days get shorter/longer?

No (not within a min or two). When days get shorter, it's not like they just lose daylight in the evening.