The open ended bug hunt is the best use case for these agents. Finding nothing costs a few dollars, finding a division by zero in ffmpeg pays for itself.
No doubt fuzzers (vibecoded or otherwise) can be powerful, but can't you just mark all "/" as potential divide by zero errors?
I guess sometimes developers think they "know" some variable won't be zero, but unless it checked explicitly or by the compiler, that shouldn't be trusted.
Generating an incorrect input file seems to be the easiest task of all for any fuzzer.
Generating correct input to get deep into the call stack and then finding something is the hard part.
The fruits of using LLMs to code. You'll waste far more time finding what it quietly and subtly wrecked than you would have if you just coded it yourself.
It’s interesting how AI may both raise and lower the quality of software. It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal. Time is much more important for a human developer with a salary.