Think of it this way, how would you turn a set into a vector in the first place? We solve this in programming a lot, for example, the "one-hot" encoding for neural networks. Here, a set turns into a vector that has a zero for every item that isn't in the set, a one for every item that is.

Now, there are a lot of things that |v| for a vector can mean. In the L1 distance you just add up the absolute value of each dimension. You could argue that that's a simpler sort of |v| than L2.

And there you go! |S| on a set actually means exactly the same thing as |x| on a vector, if you interpret sets as vectors in the right way.