Yes, insulation works both ways. My garage is unheated and insulated. If I go out there to work on something in the winter I always compare the temperature outside. On a sunny day it might be pleasant outside and freezing in my garage - so I'll open the door and let it warm up.

Insulation makes the house more resistance to temperature change (relative to the inside and outside).

One thing people forget is the delta is very different in the summer and winter. Lets say your thermostat is on 70 year round. If it is 100 degrees out you only have to cool 30 degrees. When it is 0 F out you have a delta of 70 degrees. So for this scenario, expect to use more energy in the winter.