Scientists tend to understand that part, that's more of a political/cultural thing (I'm ignoring the language part here entirely about which terms to use for which concepts here).
There's the X and Y chromosomes, those produce a binary result (unless you have a genetic anomaly). And after that comes the messy and fuzzy parts I mentioned, where those genes trigger changes in hormone levels and development. And those parts are analog, very complex and contain a lot of different parts. So the outcome is not binary anymore.
...well actually...
There are more combinations than merely having only X, or an XY combination. And there is more fuzziness even in the Y and X expression, as you said. It's fuzzy all the way down. The tale of Binary results has always been from compression of reality: Always has been.