Ice cream / milk has saturated fat, which is something that people who are watching their cholesterol might want to limit. Fat helps slow digestion, which is beneficial for the glycemic index. It depends on the ice cream too. Steel cut and rolled oats' glycemic load is not that bad, especially when eaten with berries, nuts, etc.

Also, using ice cream as the benchmark is misleading, as people might view it as a junk food and think that its glycemic index is higher than they otherwise would, but actually its glycemic index is low/moderate, depending on the type.