If you can't sell clothes to anyone, then it may be more socially efficient to destroy them than (a) keep them in a warehouse or (b) ship them overseas. Both (a) and (b) can have substantial environmental costs. I don't think it should be hard to come up with other plausible cases. You're assuming there's only one reason companies do this. I don't deny that that's a possible reason. I also don't see why taxing that behaviour would not reduce it.
Or (c) lower the price to the market clearing price point and your clothes will be gone.