This reasoning holds while open source models are (relatively) dumb.

If/once open AIs will be considerably more powerful, and runnable on consumer hardware (and we're on a trajectory for both), then everybody will have essentially a dangerous weapon in their hands (open models can be fine tuned to remove guardrails).

By the way, you're conflating two different dangers - doom scenario is a different one.