Déstabilise orbits - definitely, but the same would be true of a star or other celestial body.

But our risk of getting sucked into a black hole is really tiny. (Although personally if I had a chance to enter a black hope, I would. I’m just naturally curious.)