Zero-config usually means the complexity got shoved somewhere less visible. An SSH config is fine for one box, but with a pile of ephemeral workspaces it turns into stale cruft fast and half the entries is for hosts you forgot existed.
The port issue is also boringly practical. A lot of corp envs treat 22 as blessed and anything else as a ticket, so baking the routing into the name is ugly but I can see why they picked it, even if the protocool should have had a target name from day one.
SSH configs support wildcards, so if you couple it with a ProxyCommand you can an arbitrary level of dynamism for a host pattern (like *.exe.dev).
But yeah, everything is a trade-off.