logoalt Hacker News

_pdp_yesterday at 2:44 PM2 repliesview on HN

What is the difference between this and let's say Claude Code using something like semgrep as a tool?

Also I don't think this tool should be in the developer flow as in my experience it is unlikely to run it on the regular. It should be something that is done as part of the QA process before PR acceptance.

I hope this helps and good luck.


Replies

dolftaxyesterday at 3:06 PM

On the OpenSSF CVE Benchmark[1], Semgrep CE hits 56.97% accuracy vs our 81.21%, and nearly 3x higher recall (75.61% vs 26.83%).

On when to run it, fair point. Autofix Bot is currently meant for local use (TUI, Claude Code plugin, MCP). We're integrating this pipeline into DeepSource[2], which will have inline comments in pull requests, that fits the QA/pre-merge flow you're describing.

That said, if you're using AI agents to write code, running it at checkpoints locally keeps feedback tight.

Thanks for the feedback!

[1] https://github.com/ossf-cve-benchmark/ossf-cve-benchmark

[2] https://deepsource.com/