> your body gets used to it and it becomes a baseline

That doesn't happen for every drug. But side-effects still exist for them all, so yeah, there are several reasons to limit how much you use them.