Same here. I am going to disassemble the cheap pepper grinders I recently bought to make sure there is no plastic in the grinding operation.
I switched to bamboo toothbrushes from plastic a while ago, before de-plasticizing was really a thing. Now I'm glad I did, because plastic bristles grinding against my teeth seems like an easy way for plastic to get inside my body. The bamboo toothbrushes are pretty nice too, the bristles are soft but firm, and the handle is made of bamboo too.
All disposable grinders are going to be plastic, and likely none of the refillable ones will be since the plastic burrs only last one usage before they are all chipped off in to your food.
Mentioned elsewhere too but Peugeot (yes the car company) has been making top quality affordable pepper grinders for over a century.
The simple wooden ones last a decade or longer and cost about 35 $/€/£
Your teeth seem pretty hard though, to the point that a specialist has to go over them with metal tools every year to clean them.
Personally, I spit out whatever happens when I brush my teeth. But bamboo does sound like a more pleasant experience all the way around.