Killing addicts more quickly than creating new ones would indeed eventually lead to a decrease in drug related deaths. I would really believe this because I know of multiple people that died from ODs in a fairly short window 4-5 years and that spans a range of about 12 years of people. As in to say everyone I know age 24-36 about half of those people that were opiate users died from about 2019-2023 due to fent. All of them that I know the details of were from fake pills too, so very much related to fentanyl.
Long term you couldn't kill more than existed, asymptotically the maximum number of ODs per unit time would be exactly equal to the number created, impossible to be more.