This is the only counter argument, and in fact the consensus is that they are the fairest (they tax consumption rather than production, we tend to tax producing wealth more than spending it) if you offer rebates or lower them on essentials (bread, milk, eggs, healthcare).
They also have the effect that everybody has to pay them, including tax evaders, tax loophole abusers, criminals with undeclared incomes, etc, everybody has to pay it.
But yes, without offering sales tax rebates or with taxing essentials then your argument is true and they become less fair.
Albeit, the elephant in the room is always the definition of fairness itself.
In my eyes they are not fair, because while they tax consumption, they very disproportionately affect buying power the less wealth you have. For a millionaire, paying let’s say 30% more for new shoes is not going to meaningfully change how much money they’re left with. If I’m poor and I need new shoes (because you can’t just afford a new shoes when you want them so it by necessity implies you’re in desperate need of them), that extra 30% means one less grocery trip. Or heck, even 30% on groceries potentially means one less grocery trip.
Which is to say, being poor is expensive, and sales tax only makes it more expensive, while literally not affecting the bottom line of those in higher income brackets.