logoalt Hacker News

bhaaktoday at 7:08 PM3 repliesview on HN

It passed all the tests.

If you can't trust your test suite to catch an automatic language translation you shouldn't trust it at all. :)


Replies

user142today at 7:47 PM

Tests can only prove the presence of bugs, but not their absence. If the AI can access the tests, it can easily make them pass by just adding additional if statements. It doesn't mean the code is actually correct.

andrewflnrtoday at 8:32 PM

What if we only trusted the test suite a reasonable amount, instead of pretending trust must either be blindly total or nonexistent?

debugniktoday at 7:32 PM

It also modified many of the tests to make them pass in mischievous ways. You can't trust a test suite to catch regressions if the new version doesn't use the same test suite.

show 2 replies