Even in an environment where filming (with phones) is common and acceptable, smart glasses can still come off as rude because others find it hard tell if you are recording or not.

To record a video on your phone you need to hold your phone up pointed at the other person, usually not in the same way you would normally use a phone. If you see someone holding his phone steady at face level and pointing at something without making finger movements, you know he is filming. If someone is pointing his phone down towards the ground and scrolling around with his thumb, you know he is probably not.

To record from a pair of smart glasses you just need to look at someone, as you would normally look at any other thing. Yes there will be an LED on, but the person being recorded probably couldn't see it if it is in a bright, busy environment and you are more than a few steps away, plus there will be aftermarket modifications to disable the LED. In short, there is no way you can reliably tell if someone's smart glasses are filming you. You have to assume that worst.