Same thing as changing your ssh port to something random. It's a trade-off with the convenience of knowing that all of your servers are listening on port 22 and you won't need any customizations in scripts or whatnot. But there are ways to mitigate much of that.

On the benefit side, mitigating most of the computational load, log analysis load, how much are the baddies poking me while I sleep load, etc...all of these together make changing such defaults a slam dunk IMO.