This has to be read in terms of https://xkcd.com/2400/. If you're chasing low-hanging fruit, and searching for a 3x or 10x speedup on something that has never been optimized, this advice probably can be ok.

The harder you push, and the more you need to start finding smaller improvements, the more this advice becomes a rule of thumb you can't rely on.