This is genuinely the only response I ever get - that wealth can be used to influence politics. In my view this is a poor argument for two reasons.
1. The amount wealth actually influences politics is hard to measure but likely much lower than most people assume. Trump was outspent significantly both times he won. Bloomberg dropped $1B in a couple months and won nowhere but American Somoa. Probably the two biggest boogiemen, Koch and Soros, have spent billions over the years on their causes, and the present administration and general overton window is actually something neither of them like! The nonelected king-makers in American and EU politics are not actually wealthy people at all, just those with a lot of accumulated political capital, for instance, Jim Clyburn who single-handedly gave the 2020 nomination to Biden.
2. The amount that it takes to finance initiatives is much lower than centibillionaire level. Is the original $20B OP mentioned not sufficient to finance some ballot initiative? Why is it the increase to $130B that causes concern? The truth is that even a wealthy non-billionaire can easily do that, or bankroll someone's run for congress, or fund a partisan thinktank. The maximum level of wealth you'd have to set your ban-limit at would be problematically low.