alt Hacker NewsIt’s amazing how often an LLM mocks or stubs some code and then writes a test that only checks the mock, which ends up testing nothing.
It’s amazing how often an LLM mocks or stubs some code and then writes a test that only checks the mock, which ends up testing nothing.
You really do have to verify and validate the tests. Worse you have to constantly battle the thing trying to cheat at the tests or bypass them completely.
But once you figure that out, it's pretty effective.