Yeah, it makes more sense to put all the fragile body parts in one place. For instance, the face.

I read this does have a functional reason: Sperm cells have to be kept at slightly lower temperature than the body temperature, so if the testicles were inside the body, the sperm wouldn't survive.

Of course you could ask why sperm is so temperature sensitive in the first place...