alt Hacker NewsThere is a ton of questions to ask regarding their removal of cybersecurity data.
First, how much “cybersecurity data” is just standard software knowledge that one can derive cyber security exploits from?
Did they also get rid of all references to hacking in general?
Are they implying that all the recent exploits are just things that were already in the data?
For a model it doesn’t seem that far fetched to make the leap from “I have the knowledge of what hacking is” and “I know how computer systems work” to just being able to exploit stuff anyways.
Replies
If you're an amazing, top 1% software engineer, and you switch jobs to become a penetration tester, you're not going to be top 1% day one. There's knowledge specific to penetration testing you'll need to research and absorb and practice before you're good.
For the models, it's always day one. They don't learn from experience. Whatever they know after training is all they'll ever know. So if you don't teach them penetration testing in training, they won't know it. Maybe they could theoretically derive everything from first principles, and so could a top 1% software engineer, but that's a very slow and inefficient way to do things.
> For a model it doesn’t seem that far fetched to make the leap from “I have the knowledge of what hacking is” and “I know how computer systems work” to just being able to exploit stuff anyways.
You'd be surprised. Cyber offense is a very narrow domain, especially given the extensive mitigations that are being used in modern systems. Leaving it out of post-training tasks unless you're specifically interested in it is the no-brainer approach.
I have essentially no knowledge in this field, but another concern may be that if the model has minimal cybersecurity knowledge, it may also implement insecure systems as a side effect.