Sure, it is quite far-fetched. However it is extremely uncommon that we experience unified social trends all across the board, from liberal Finland or Japan to North Korea and Taliban-run Afghanistan. Usually there are odd reversals and exceptions here and there; not this time apparently. And we still lack a satisfying theory that could account for fertility decline in every country.
We do. It's called "urbanization".
Large cities are inherently inimical to living in large families. And yes, it was apparently the case even in the Roman Empire.