That's true for just about any heuristic-based optimization - your assumption breaks, and the optimization with it.

That doesn't make it useless, not at all. It's still useful in its regime, and you can work around to cover the gaps where it breaks down.