In general I agree, but it's worth noting that rich people are also very interested in finding tax loopholes.

I heard that, at least in the US, you can avoid capital tax gain by just... never selling. Borrow against your wealth instead.