No idea where you're getting it from, Germans are Nemci in Russian as well. It's rather "unable to speak the language", meant for all foreigners but later stuck to Germans, presumably because German traders were the most common foreigners.
> Germans are Nemci in Russian as well
I wanted to check; are you implying that Russian is not a Slavic language?
Apologies, it was mostly from running across different Russian maps with Германия that I took it as such (in Serbian it is Немачка). I stand corrected!
Nem/нем literally means "mute" in Serbian, perhaps it's a latter evolution per region either way.