> My take would be that underwater temps cannot get below freezing

Yep, density of water changes with temperature—it's the only thing that has that phenomenon—and therefore the colder water is always at the surface. That's actually what made life possible since life could survive despite a frozen surface.

Happy to be proven wrong, but that's what I learnt making cocktails and investigating clear ice :)

plus thermal conductivity of water vs atmosphere

Freezing point of water changes with salinity and impurities. By definition a liquid cannot be below its freezing point until it has completed its phase change.