Because retailers take the legal responsibility for what they sell to the public. In the same way statutory returns go to the retailer, not the manufacturer (unless the manufacturer has volunteered an extra warranty to use on top). They can take it up with the manufacturers if they want.
The customer doesn't enter into a contract with the manufacturer when they buy an item from a retailer. They do so with the retailer.
Which is not to say the agencies shouldn't also ban the product from the market. But that doesn't absolve the retailer of their duties.
I take it you're not in the US? Retailers here generally must warrant against DOA, but almost anything beyond that can absolutely be shoved off onto the manufacturer.
Those who wonder why Americans can get so many things so cheaply - yes, we have lower tariffs (or at least did), but also we don't have those minimum-duration warranties that allow the consumer to return to the place of purchase as much as a year or more after purchase and demand satisfaction from the retailer. Those are expensive to provide.
This feels gross. If the manufacturers are not held accountable, then they will never stop making shite products. If the retailers are knowingly selling shite products, then sure. But most of the time, retailers are just selling products they think customers will want to buy. Lots of shite products get returned to the manufacturer when the retailers cannot move them, or other issues arise from the products.
The retailers are the front line for returns as that's the point of contact for the customer. That's just the nature of the beast. If a company establishes itself as a maker of decent products that retailers can trust suddenly gets lazy/cheap/profit focused to the point they cheapen their products, it is not the retailer's fault.