Populations can grow, even rapidly, in the short run even with collapsed fertility. This is because in part it's a lagging metric, but more importantly (and less intuitively) because fertility also determines the age ratios within a society. Imagine a society has a common fertility rate of 1 then that means each successive generation is half as large as the one prior. So if there's 1 new born, then there's 2 20-40 year olds, 4 40-60 year olds, and so on. High fertility rates result in societies with far more younger than elderly populations, and vice versa for low fertility societies.

In the past most societies had high fertility rates, which means there are relatively few elderly, relative to younger people, today. So the global death rate is relatively low as the the number of people annually exiting the stage is relatively low. But when low fertility generations start to die off, this gets turned upside down. You'll have large numbers of elderly dying off, and they'll make up the majority of the population. But this only happens ~80 years after the birth of the low-fertility generation.

This is why Japan looks so catastrophic today even though their fertility rate is much higher than places like South Korea. They've had a mediocre fertility rate since the 60s which then went into collapse in the 70s. By contrast South Korea had a fertility rate of 6 in the 60s, and didn't fall below replacement until the 80s. So Japan has had a solid decade or two head start. The fact we still aren't at the life expectancy of the first below replacement generation in Japan also means today is still going to be seen as 'the good ole days' for them. And this is where most countries are headed.