logoalt Hacker News

cpercivatoday at 5:43 AM1 replyview on HN

I've seen plenty of people saying "Mythos isn't all that exceptional, lots of LLMs can find security vulnerabilities" -- and indeed there is some evidence for that; it sounds like Anthropic was taken somewhat by surprise at how easily a simple prompt managed to get Mythos to deliver exploits and didn't distinguish immediately between the effectiveness of Mythos and the effectiveness of the prompt.

But the claim of "LLMs aren't making a difference in vulnerability discovery" has been laughable to anyone who has been reading security advisories for the past 3 months. Just look at the Credits lines.


Replies

wrstoday at 6:21 AM

I thought the point was not that Mythos finds more vulnerabilities, but that it can exploit them much more successfully. I thought the report showed it didn’t find much more than Opus 4.8. (Or did I misread?)

show 1 reply