I dunno I think it makes sense. For a vector x, the length |x| says something about its size relative to other vectors. For a set S, the cardinality |S| says something about its size relative to other sets.

The vector is always defined in a vector field which has a given dimension, and usually the dimension isn't that interesting. Typically it's either the same between the vectors you consider, or the vectors have one of a few fixed number of dimensions. Meanwhile the length of vectors is an interesting quantity.

For sets, since the values can be anything, nothing or everything in between, you can't really define many interesting functions or operations that work on the elements of sets in general. Meanwhile, the number of elements in a set is an interesting quantity.

Anyway, just my take, though I never did take much math.