alt Hacker News> The shortest path for the model is to implement it completely
Have you worked with LLMs??????????? “I disabled the test so it’s not run so now all the tests pass” is not a hypothetical it’s pretty common. LLMs frequently do shortcut learning. The reason why reviews are expensive is because you still need to do all the steps in order to understand if a shortcut is justified.
Replies
I used Fable to write a relatively small RPG. In the span of 2 hours it managed to do many interesting things. My favorite was when it wrote code with a race condition that could cause people to take more damage than they should, which it then defended as an acceptable tradeoff for parallelism.
> “I disabled the test so it’s not run so now all the tests pass” is not a hypothetical it’s pretty common.
I've never seen that though.
> “I disabled the test so it’s not run so now all the tests pass”
Also:
"I implemented it this terrible way because of precedence in the codebase...that I just wrote"
"I avoided implementing this correctly because of migration concern for existing installations of this code I'm writing right now"
"I deferred this critical feature for the future, so we can deploy quicker"
or, my favorite,
"I hand rolled an buggy http server because you said the tool should be self contained"