alt Hacker NewsOn my last project, it kept trying to use the system python instead of the project's virtual environment. It also kept using the wrong build tool. Both things wasted considerable tokens because the agent got sidetracked trying to understand why it could not run the tests - and that repeated on each new session.
A simple instruction in AGENTS.md fixed that.
Replies
This is what the majority of mine look like as well. Just simple instructions for things I need to do repeatedly.
I have not even really needed a formal memory system. If I see an error happen more than once, I just say "hey add a note on this to agents.md". Tends to be verbose but overall works quite well for the projects I am doing.
Can you share which model and harness created this situation?
Yeah, I feel like what these files do is not that difficult to understand; it's just context that the agent will pick up and use pretty much the same as any other context it has. No, it won't deterministically prevent things with a static check, but it will work about as well as just manually telling the agent "don't do X" as part of the prompt. The fact that it's a "mostly works" mechanism rather than a "guaranteed to always work" mechanism is pretty much the same experience that using an LLM gives in general, and while that requires a bit of thought about how to use it, it's still good enough to be useful for a lot of things.
Before LLMs, I found "mostly works" systems like this to be incredibly sketchy and not worth using. The main thing I've had to learn in this past year is that what I thought was an ironclad rule turned out to be only a heuristic that was useful before but not always helpful, because empircally as much as I might find the lack of determinism jarring, in practice these tools are genuinely good enough at what they do to be worthwhile to use, as long as you're making sure not to use them in ways that the occasional failure costs more than just some wasted time.