Indeed, much of the scariness is how fearlessly and confidently it writes them with little regard to their actual usefulness or value. When I find it adding a lot of tests, I often say something like: "audit each test carefully, and consider whether the test is testing a meaningful boundary or is more ceremonial. delete low-value tests and add new tests to cover meaningful boundaries not exercised by the gaps you identify". Without fail, this always produces some decent results.
Having said that, in truth, I almost never read the unit tests. Before AI, we had almost none (see: several person game studio) so the tradeoff is not "AI-generated tests" vs "human written ones", it's whether we have tests at all. So, I take them for what they're worth - not much - but if it catches an extra regression before it ships every now and then, it was worth it for the price (~free).
alt Hacker News
Indeed, much of the scariness is how fearlessly and confidently it writes them with little regard to their actual usefulness or value. When I find it adding a lot of tests, I often say something like: "audit each test carefully, and consider whether the test is testing a meaningful boundary or is more ceremonial. delete low-value tests and add new tests to cover meaningful boundaries not exercised by the gaps you identify". Without fail, this always produces some decent results.
Having said that, in truth, I almost never read the unit tests. Before AI, we had almost none (see: several person game studio) so the tradeoff is not "AI-generated tests" vs "human written ones", it's whether we have tests at all. So, I take them for what they're worth - not much - but if it catches an extra regression before it ships every now and then, it was worth it for the price (~free).