This seems anti democratic. How can we prevent small minorities from hassling everyone until they get their way?
What if the small minority is being oppressed and killed? There are so many reasons why a small minority might need to protest within a democracy. "This seems anti democratic" is a bizarre take.
In the US? You'd probably need to repeal the first amendment. Good luck with that.
In the big picture, culture progresses towards equality over time, although it see-saws and moves very slowly relative to a human lifespan. Small minorities of hate groups, for example KKK, are not able to influence society in the long-run because their message is antithetical to this natural imperative. Whereas advocacy for racial minorities, gender minorities, and feminism progresses over time.