I am inclined to agree. The issue is that we are humans and not all of us play nice. And sometimes, even when we play nice, things happen. I am not big on guardrails, because in US they have turned into yet another cottage industry and I fully expect an association credentials popping on linkedin soon. What this means in practice is that it is never enough. Whatever the current state is, the association will be pushing towards yet another another extreme.

There is an argument to be made that there are a lot of not great people out there, who may abuse tech, but the response should be not be: kneecap said tech. The response should be: smack those people's hands. I don't think anyone will actually complain if police catches someone, who is looking up poison recipes.

What I do think, however, that reasonable people will complain when we move to the pre-crime territory ( you saw him looking up poison recipes and did nothing! ) and show up at your door to inquire about your llm prompts. To me it is an issue.