Says more about the relatively poor infosec on etherium contracts than about the absolute utility of pentesting LLMs.
True, I'd be curious to see if (and when) those contracts were compromised in the real world. Though they said they found 0 days, which implies some breaches were never found in the real world.
4.6M is not a lot, and these were old bugs that it found. Also, actually exploiting these bugs in the real world is often a lot harder than just finding the bug. Top bug hunters in the Ethereum space are absolutely using AI tooling to find bugs, but it's still a bit more complex than just blindly pointing an LLM at a test suite of known exploitable bugs.