This phrase only made sense when it was explained to me that it’d be better phrased as “the exception that proves the existence of the rule”.

(if there were no rule, there wouldn’t be any exceptions to it, or nothing would seem exceptional with respect to the (previously unstated) rule)