This matches my intuition. Zero is synonymous with "the absence of any X".

The singular equivalent would be perhaps "non-" or "-less".

Hot take: zero is a math concept and math deals with multitudes only (even under one, you're dealing with a multitude of parts). The actual irregularity is the usage of singular noun form in a math context.