It's mostly historical and a combination of security from the WW2 and Cold War eras as well as how the gold standard worked before Bretton Woods.

Most countries held gold reserves as their currencies were backed by it. When money needed to move because of trade, the ratios of currencies backed by gold would shift as money got converted to other currencies. However, constantly moving physical gold back and forth with the ebs and flows of trade was impractical, so ownership of gold in central bank vaults usually just had its legal ownership marked on ledgers. Physical gold moving was more part of reconciliation. This description is an oversimplification as in practice, countries didn't always play by the rules of the gold standard (revaluations, going on and off the standard, etc), but you get the general idea.

This mostly ended as Bretton woods came into effect and the new system was basing the US dollars on gold, then other countries held direct US dollars and traded that instead. Countries could then convert those US dollars into gold. This ended when Nixon closed the gold window when he refused to raise interest rates as there started to be a run on gold as countries rushed to convert. The backdrop was that high spending on the Vietnam was and new social programs was driving inflation, but Nixon knew that would cause an economic slowdown he wasn't willing to do. Jimmy Carter then took a lot of the political flack for allowing the central bank to raise rates to kill it. This is why central bank independence is so important.

During times of geopolitical risk, countries would physically move their gold to be held in trust in friendly and/or rule of law countries. The USA was seen as this between ~1914-now.