I would agree, except for some factors:

Gravity tends to bring heavy objects together

Andromeda and the Milky Way are not colliding "like two pancakes on top of each other" but at an angle to each other

Also star surroundings are usually places where you can collide with a lot of stuff (asteroid belts, dust, etc)

We're not shooting a goal on a 4 light-year wide goalpost (btw that's the density on our vicinity, but on other areas the density is higher) but passing multiple stars on an environment that can be perturbed and heavy things attract each other