Ok, I guess I'm always confused why |S| on a set doesn't take the values into account, whereas |x| on a vector does take its values into account; how can mathematicians of all people be so inconsistent?

This is extremely common notation, and it’s common across STEM fields. If you’ve done any discrete math, algorithms analysis, etc. this should be familiar to you.