counter-point: having to constantly track git status in your head, and needing to type commands to remind yourself, is a far bigger distraction. Optimize to avoid context switching, not for a few ms latency.
FWIW, I switched from zsh default to starship and didn't notice any perceptible difference. But I certainly notice when I mess up my git commits!