alt Hacker NewsI think this brings out the cognitive dissonance around "safety" regarding cyber security:
a) In order to make us safe, the LLM should help us find (and fix) the vulnerabilities in our own code.
b) In order for us to be safe, the LLM should not find vulnerabilities in other people's code.
I don't think this is resolvable in a way where both (a) and (b) win.
Exactly, it's a failure of Anthropic and others to understand cyber security. Finding security bugs in software is a good thing and not evil. It will lead to more secure software.
Defense and offense in cyber security are two sides of the same coin.