It's worked in the past. But it does require someone at your org to care that CI times are spiking, which is not always a thing you can rely upon.

In addition: if CI is the only place the issue shows up, and never in a user interaction... Why does that software exist in the first place? In that context, the slowdown may be serving as a useful signal to the project to drop the entire dependency.

ETA: To be clear, I don't do this as a substitute for a regular deprecation cycle (clear documentation, clear language-supported warnings / annotations, clear timeline to deprecate); I do it in addition before the final yank that actually breaks end-users.