logoalt Hacker News

jasonlotitotoday at 2:59 PM1 replyview on HN

> I've yet to find a way to get an LLM to actually follow the instructions in CLAUDE/Agents.md

I'll give you a hint: do you rely on people to do the right things, or do you have automated unit, integration, and e2e testing? Do you have linters? Do you have static analysis that automatically runs and will block when violated?

If you are verifying humans, why aren't you verifying LLMs?


Replies

maccardtoday at 3:33 PM

Yes, and agents are generating code that fails those checks regularly. I wouldn’t accept a junior engineer saying “oh I just submit my code and see if the tests pass, let me know if they’re failing and I’ll fix them”. Pre commit hooks are disable-able and LLMs have shown they’re more than willing to say “tests were failing before this change” when they weren’t, or to use python scripts to self escape their sandbox restrictions.