New knowledge doesn't necessarily push out old knowledge, and we probably don't have infinite capacity for knowledge. That being said, at least in my experience, the time when new pushes out old is when old is less useful than new.

Retaining (again just speaking for myself) requires actually using / applying the knowledge at some point within some timeframe of learning it. Otherwise yeah it fades to the point of disappearing over time.