alt Hacker NewsI'm still trying to understand what a subnormal number is; IE, I'm looking for the TLDR so I know just enough to know if I'm using them and need to learn more.
Unfortunately, the Wikipedia article, while probably being accurate, doesn't give a clear and concise answer.
IE, is 0.0001 a subnormal? Or is it 0.000000000000000000001?
Replies
Usually IEEE floats have an implied 1 in the front. So for the standard represented numbers, there's some minimum number 1.bbbbbb.. * 2^-N. This allows 1bit more precision than is actually stored.
between any two numbers, there's basically the same epsilon difference, but from the smallest number to zero it's bigger.
A subnormal number breaks that convention, it just becomes 0.bbbbb... * 2^-N. As the numbers get smaller, the relative difference between the numbers gets larger. That also means their precision is smaller than the normal floats.
Floating point numbers are usually interpreted as
where sign, mantissa and exponent are fixed bit width integers. The 1. before the number is normally implicit because it would be a waste of a bit to encode it when you could just use a diferent exponent to represent such a number.However with this simple scheme the number zero and a relatively large gap around it cannot be represented (relatively large to the gap between the smallest and next smalles number that can be represented).
So there is a special case where for the smallest encodeable exponent the mantissa must also specify that 1. or 0. prefix. Because its a special case it needs special handling that clever silicon engineers might think is unimportant enough to handle in microcode instead of dedicated silicon.
x86 has a mode to assume that all such small numbers are actually equal to zero which can then be handle without microcode fallback. Technically its even a bit more complicated because x86 has two different float implementations and for at least SSE floats you can control the denormals-are-zero and flush-(denormals)-to-zero-(when writing) modes independently. GCC -ffast-math actual enables that mode for the entire main thread.
AFAIK ARM NEON always works in that mode so the Gravion and Apple benchmarks might be unfair here undless you compare with DAZ and FTZ enabled on Intel. No idea if the AMD benchmarks might have used different modes. Because the flags are global per thread you can easily have unrelated loaded libraries messing the benchmark up.