In general from a formal logic perspective the whole idea of “an exception that proves the rule” is flawed. If the statement was “an exception that disproves the rule”, then I would agree.
I always thought it meant "the exception (to the rule-of-thumb) proves the (hard, correct) rule".
It's only flawed because you are also using it wrong!
"The exception that proves the rule" does not mean that an exception confirms a rule in a logical sense. Instead, it originates from legal and linguistic contexts where an explicit exception implies the existence of a general rule. E.g. a sign that says "No parking on Sundays" implies that the rule is that parking is fine on other days.