This argument never really holds water with me, why stop at card costs? Why not add on an additional fee for every other thing like, fuel surcharge on delivery costs, employee surcharges (I don't live in a place where tipping is expected), or even energy surcharges?
The cost of the product should be what's on the label. If that's not enough, raise the price.
Because, between all of these expenses, it is the one the customer controls. The customer chooses what card to use, but the merchant pays the associated cost.
The credit card fee is the only thing that changes based on the payment method. The fuel cost, employee cost, etc. is the same for the same product, but if you pay with a credit card, it costs the seller more money.
It's common for the price you pay to change when the method of purchase changes. For example, having a product delivered costs the seller more than if you picked it up from the store, so you have to pay a delivery fee.