logoalt Hacker News

jatinstoday at 12:52 AM3 repliesview on HN

If Agent is writing the tests itself, does it offer better correctness guarantees than letting it write code and tests?


Replies

bluGilltoday at 1:04 PM

In my experience the agent regularly breaks some current features while adding a new one - much more often than a human would. Agents too often forget about the last feature when adding the next and so will break things. Thus I find Agent generated tests important as they stop the agent from making a lot of future mistakes.

MillionOClocktoday at 2:46 AM

It is definitely not foolproof but IMHO, to some extent, it is easier to describe what you expect to see than to implement it so I don't find it unreasonable to think it might provide some advantages in terms of correctness.

show 1 reply
rvztoday at 4:44 AM

Given the issues with AWS with Kiro and Github, We already have just a few high-profile examples of what happens when AI is used at scale and even when you let it generate tests which is something you should absolutely not do.

Otherwise in some cases, you get this issue [0].

[0] https://sketch.dev/blog/our-first-outage-from-llm-written-co...

show 2 replies