alt Hacker NewsIf a model fails the test, it should be banned. He is not advocating a ban of open-weight models. He is advocating a ban of models that fail mandatory safety testing. Seems reasonable and straightforward.
Replies
Any sufficiently capable open weights model would fail "safety" testing though, as any "safeguards" of the sort Anthropic likes could be removed. That's just another way of saying they want a ban on capable open source models which would contradict their earlier statement, or at least make it very misleading. It's hard to see how this post can be internally consistent without some hint from Dario about what he believes should happen to models that fail safety testing and/or how capable open weights models could possibly pass a safety test of the kind he proposes.
Can you think of anything open-source that has to go through mandatory tests to be distributed and still survived ?
There is a reason to it, that's as good as any angle to find why IMHO.
He is not advocating for banning models per se but the proposal makes a business model (i.e. serving open weight models) that is starting to work more expensive.
abliteration and fine tuning makes it not so straightforward
> He is not advocating a ban of open-weight models
He is though. He wants open weight models banned that do not pass some set of tests.
And what does "safety" mean here? We constantly see these companies treating NSFW content as "unsafe", despite the fact that its not. Is being able to produce adult content going to result in a model being declared "unsafe"?