The ice grinding primarily happens at the base of glaciers, in valleys.

Above the bergschrund (the head of the glacier), erosion in high mountains is accelerated by freeze-thaw cycles. Temps above freezing obviously contribute to this. But even well below freezing ambient temps, exposed dark rock in sunlight can absorb enough energy to cause local thawing, which results in rockfall.