There's actually (at least) three things going against you going short:

- position has significant negative carry (what you're talking about there)

- stock/bond prices are nominal and the government constantly prints the denominator so prices tend to go up even if there's no actual growth

- for equities there is a genuine long term positive drift over time even if the denominator doesn't change

So yes, it's hard to make money going short and timing is everything