Yep, Stochastic Rounding [1]. I actually had "round-to-nearest" in the title of the piece originally. Usually people do that if they don't have the space for fp32 master weights, but it doesn't fix the second moment part of the learning process.

[1] What is New in Stochastic Rounding: a Survey on Theory, Hardware, and Applications, https://arxiv.org/pdf/2603.06060v2