Fixed-point arithmetic provides uniform absolute precision; floating-point arithmetic provides almost-uniform relative precision, as used by scientists and engineers for literally centuries (“significant digits” except the binary version is more uniform than the decimal one customary with pencil and paper).

Hardware floating point on CPUs (including SIMD units) is almost always IEEE 754 compliant these days (excepting only IBM’s weird fantasy land), and there the rules for the non-YOLO operations (+, -, *, /, sqrt, fma) are completely unambiguous and deterministic: treating the inputs as exact, compute the mathematically exact result, then either return it as is if it is exactly representable as a floating-point number, or if it’s not then round it to one that is according to the current rounding mode.

Things that can mess this up:

- GPUs just do whatever they feel like will make them look faster on benchmarks, don’t count on anything.

- Transcendental functions (exp, sin, etc.) are really hard (multiple literal PhDs) to implement according to the rules I’ve just described (“correct rounding”), so you’ve only been able to get such implementations in the last few years, and I believe no stock libm has completely switched so bring your own if you need them.

- Decimal-to-binary and binary-to-decimal conversions, by contrast, are not that hard to implement according to the rules in principle (it’s making them fast that’s difficult), yet Microsoft couldn’t get it right for literal decades, so if you need Windows then double-check CRT versions and bring in well-known open-source conversion code as necessary.

- Denormal inputs or outputs are very slow in some implementations, leading to a hardware option to flush them to zero. Either make sure to not produce them or keep an eye on the option.

- The precise bit pattern of the NaNs you get for invalid inputs may differ across platforms. Either make sure to not produce them (you really shouldn’t) or canonicalize upon de/serialization.

- Sometimes compilers will try to HALP by performing e.g. single-precision math in double-precision accumulators and only rerounding upon store to memory; by fusing * followed by + (two roundings) to hardware fma (only one); by reassociating; etc. Take care to prohibit your compiler from doing these shenanigans (no -ffast-math or -funsafe-math-optimizations ever, in your code or in any dependencies, and God help you if you’re on MSVC).

- Most shamefully, the 8087 (despite spawning the entire IEEE standard in the first place) tried to HALP by using 80-bit registers, so if you need x86-32 then be especially careful with compiler settings (I seem to remember the HALP mode might even be ABI-mandated on some 32-bit platforms so you’ll need to violate that).

The concept of floating point is solid, the IEEE standard is stellar, but the superstructure around it is just—not, requiring an unnecessary amount of vigilance to just make it work as designed.