alt Hacker NewsTon of work already being done on this. I am a Vulnerability Researcher @ MIT and I know of a few efforts, just at my lab alone, being worked on. So far nearly everything I have seen seems to do nothing but report false positives. They are missing bugs a fuzzer could have found in minutes. I will be impressed when it finds high severity/exploitable bugs. I think we are a bit too far from that if its achievable though. On the flip side LLMs have been very useful reverse engineering binaries. Binary Ninja w/ Sidekick (their LLM plugin) can recover and name data structures quite well. It saves a ton of time. Also does a decent job providing high level overviews of code...
Replies
I definitely agree that there's a lot of research happening in this space, and the false positive issue is a significant hurdle. From my own research and experimentation, I have also seen how challenging it is to get LLM-powered tools to consistently find real.
Our approach with Jazzberry is specifically focused on the dynamic execution aspect within the PR context. I am seeing that by actually running the code with the specific changes, we can get a clearer signal about functional errors. We're very aware of the need to demonstrate our ability to find those high-severity/exploitable bugs you mentioned, and that's a key metric for us as we continue to develop it.
Given your background, I'd be really interested to hear if you have any thoughts on what approaches you think might be most promising for moving beyond the false positive problem in AI-driven bug finding. Any insights from your work at MIT would be incredibly valuable.
Agree with you on that. There is nothing about LLMs that makes them uniquely suited for bug finding. However, they could excel re:bugs by recovering traces as you say, and taking it one step further, even recommending fixes.
We largely agree, we don't think pure LLM-based approaches are sufficient. Having an LLM automatically orchestrate tools, like a software fuzzer, is something we've been thinking about for a while and we view incorporating code execution as the first step.
We think that LLMs are able to capture semantic bugs that traditional software testing cannot find in a hands-off way, and ideas from both worlds will be needed for a truly holistic bug finder.
The funny thing is…
You’re stating the problem with that whole sector.
I really wish product owners, researchers and tool creators had more actual real world experience on the remediation side. I think that’s the reason we have so many crappy tools.
- We need a better way of addressing business logic issues and sensitive data leakage which starts at the data model and flows from there.
- Within large organizations we need better risk data about vulns to aid with prioritization and remediation which is always the larger problem (sifting through noise)
- We need automated threat modeling tools that reduce a teams need to start from zero
Fundamentally a tool is a waste of time if it can’t tell you there’s “x% possibility of downtime or sensitive data leakage.”
Addressing the risk equation (r=il) where the impact and likelihood variables are baked into every tool and based in real world data is where we should be.
Until then, vulnerability scanning and management will continue to suck.
Solving the false positive problem is like solving the halting problem. I don’t think we get to a world where static analysis tools don’t have them, AI or otherwise.
That said, I have found LLMs can find bugs in binaries. It’s not all false positives, as far as I can tell. I have a side project I’ve been working on that does just this (shameless plug): PwnScan.com. It’s currently free and focused on binaries.
The bad news is that you quickly get into a situation where you have too many false positives where it’s sometimes not feasible to sort through them all.