If we are talking about amateur sports where the stakes are low, the concept of slotting athletes into divisions makes sense.
In elite sports, no one wants to see "best in division X". They want to see the best hockey players, the best golfer, the best skier, etc. The money incentives are considerable.
This would destroy women's professional sports.
Implicit in what you're saying is that they want to see the best sex-identified athletes in a given sport. If that wasn't true, women's sports would have no audience and we know now (finally!) that this is not the case.
I personally think that we'd live in a much better world where you compete against others who broadly speaking are in the same performance category as you.
But I do appreciate that the transition to such a world would, indeed, destroy women's professional sports, and thus I do not attempt to really advocate for that transition. If it could happen overnight (it cannot), perhaps I would, but that's not where we live.