alt Hacker NewsInteresting, I haven't tried tests outside of the code base the LLM is working on.
I could see other elements of isolation being useful, but this kind of feels like a lot of extra work and complexity which is part of the issue...
Interesting, I haven't tried tests outside of the code base the LLM is working on.
I could see other elements of isolation being useful, but this kind of feels like a lot of extra work and complexity which is part of the issue...
The way I do it is write tests, then commit just the tests. Then when you have any agent running and generating code, before committing/reviewing you can check the diff for any changes to files containing tests. The commit panel in Jetbrains for example will enumerate any changed files, and I can easily take a peek there and see if any testing files were changed in the process. It's not necessarily about having a separate codebase.