It is crazy to me how every comment here is basically "yeah, screw safeguards".
How people don't understand the reasons for safeguards, alignment, and the ramifications of releasing more and more powerful open-weight models being an act of irreversible proliferation, is beyond me.
I wish the safety and alignment teams at OpenAI and Anthropic were allowed to post about the things they have to deal with. I'm confident their stories would be eye-opening...
What people don't like to see is their perfectly benign workloads being impacted.
When they see Fable shit a brick on basic biology questions, or Opus refuse to map out a game executable to wire a mod loader into it, or Astra refuse to red team and search for critical security bugs in their own code? They don't think "ohh, this must be so helpful, requests like mine really should be stopped." They think "this is a corporation that decided, based on arbitrary criteria, that my use case should not exist, and used a product I pay good money for to force their decisions down my throat."
Anthropic's reckless "safeguarding" and their Apple-like "we always know what our users need better than our users do" attitude have done more for the pro-open-weights, pro-proliferation agenda than just about anything ever done by anyone else.
People are now getting excited for Chinese labs to find new ways to distill, because that way, they can get Fable capabilities in a model that wouldn't arbitrarily ban REA. This is how you take the goodwill you earned for years and burn it all down.
The arguments used to support safeguards could easily apply to something like speech.
The whole idea of safeguards is that you the user can't be trusted with the power.