If you don't _need_ subnormals MXCSR.DAZ/FTZ (which you can get gcc to set via -mdaz-ftz) will let you ignore all of this.