Do you use curb cuts? Closed captions? Difficulty sliders in games? An electric toothbrush? Audible crosswalk signals?
All of those have significant roots in accessibility for people with disabilities. I guarantee you that the people who invented them would be thrilled to see them have widespread adoption for all populations.
If something finds use in addition to its use for disability amelioration, it becomes more widespread and normalized. When it's wider spread and normalized, it becomes easier for people with disabilities to know it's available and to use it without stigma.
So no, you've got it entirely backwards I'm afraid. We do not think about assistive technology as something for people with disabilities. We think about it as something that helps people, and if it helps more people, even better.