alt Hacker NewsBut no matter how conservative they make the anti-jailbreaking, the risk doesn't go away. There are so many logic "holes" that are ambiguous and can blur the line between a jailbreak and legitimate use.
If every time a jailbreak is discovered, the model has to be turned off and jailbreak prevention updated, the effect will be the same regarding how willing users are to adopt it.
Also this falls into the "right to bear arms" thing: if LLMs are limited legally, then illegal LLMs will be the superior choice. This is pretty much the plot of Cryptonomicon and Corey's take on I, Robot