The problem is that math is not some universal language, it is a broad field with various sub domains with their own conventions, assumptions, and needs.

Polysemy vs Homonymy vs Context Dependency will always be a problem.

There are lots of areas to improve, but one of the reasons learning math is hard is that in the elementary forms we pretend that there is a singular ubiquitous language, only to change it later.

That is why books that try to be rigorous tend to dedicate so much room at the start to definitions.

Abstract algebra is what finally help it click for me, but it is rare for people to be exposed to it.