On the other hand, there is just so much less stuff up there, as reaching that orbit is much more expensive in terms of energy expenditure, and it's all moving in pretty much the same direction and in the same orbital plane.

So unless somebody maliciously launches e.g. a bunch of ball bearings in the same orbital plane but opposite direction, the chances of "wrecking GEO" seem much lower (although the consequences would, as you say, probably be much more severe and long-term).