logoalt Hacker News

rafram • today at 8:17 PM • 0 replies • view on HN

The "CVE Rediscovery" benchmark chart feels... totally meaningless. Quoting the description:

> CVE Rediscovery is an internal cybersecurity evaluation that assesses how many published CVEs (Common Vulnerabilities and Exposures) a model can independently find when given a snapshot of the project before a CVE was fixed. Our October 2026 run against 1,829 CVEs showed Claude Mythos 5.2 outperformed Claude Mythos 5.1 and Claude Opus 5.5 at every effort level.

> published CVEs

So by definition this is just measuring how many CVEs (and/or explanations / PoCs of CVEs) the model has memorized. What are we supposed to learn from that?