logoalt Hacker News

baq • today at 4:29 AM • 3 replies • view on HN

It doesn’t follow. Everyone sane has the models review the choose the models wrote. Reminder these are the models which found the Jacobian and Navier-Stokes counterexamples; they’ll find holes in their own slop, too.


Replies

layer8 • today at 6:46 AM

These counterexamples are comparatively straightforward because the input domain is well-defined and simply-structured, and a counterexample is trivial to verify. The same is not true for arbitrary vulnerabilities.

➕ show 1 reply
NoPicklez • today at 4:36 AM

Yes and no, many people don't have models review the code the same way many humans don't review their own code in depth for security vulnerabilities.

Furthermore, you need to make sure the model you use is capable enough to review your code comprehensively enough. That includes for both basic vulnerabilities but also attack chain related vulnerabilities.

mepiethree • today at 4:31 AM

Then a new model comes out two weeks later and finds a hole that your archaic review bot missed

➕ show 1 reply