FWIW, I've smoked cannabis daily for maybe 15 years, I'm not exactly sedentary (have dogs, a partner and like to (lazily) swim in the sea) but generally don't exercise. Visited a cardiologist like two months ago and have perfectly fine heart despite the smoking.

Anecdotal of course, many could probably bring up counter-stories too, but I do think you bring up a good point, it seems to me a completely sedentary lifestyle seems to be way more destructive to your health than moderate usage of various drugs and/or eating habits. People who just walk a bit daily already seem way healthier and happier than peers in their same age.