Back in the 1960s (!) the military would fly supersonic over the town I lived in. I always enjoyed the boom. I would also open the window during a thunderstorm or go sit on the porch because I enjoyed the incredible electric display and the booms.
Memories of a kid on a road trip...Interceptors taking off low, at full afterburners, in the dark of the night, screaming across the highway, for their practice sorties...
I am more anti-(manufactured)-war than most, but these sights were the stuff of goosebumps (the good kind).
Yeah, I often have military flight training over my town, and I don't find the sonic booms particularly bad. I barely notice the planes unless they fly close to the ground over my house.
I'm not sure how the sonic booms of fighter jets compare to passenger planes, though.
Back in the 80's the British air force would fly ridiculously low over the rural village I grew up in, certainly lower than they were meant to be allowed to - everyone hated it. You'd be minding your own business and then just about shit yourself out of nowhere!
Military gets away with things that private industry doesn't. Supersonic passenger planes would be flying every day on a schedule, far far more than military. Commercial planes are also rated for far more flight hours.