There have been rumours like this about the US gold storage for decades.

Its audits only look at a tiny percentage of the holdings at a time, any kind of weighing or purity inspection is the exception rather than the rule, and third-party auditors or observers are not welcome. They basically look at a handful of bars, see that the seals on the other vaults are intact, and extrapolate that all the gold is perfectly fine.

In other words: the US could be pulling the fraud of the century here, and we'd never find out. But a full audit would be extremely expensive, there's no sign that there's any kind of foul play, and other countries are doing more-or-less the same.

And even if a country does notice that something is off with their US holdings, there's plenty of reason to keep it quiet and just play along.