logoalt Hacker News

StanAngelofflast Monday at 5:03 PM1 replyview on HN

I did my usual thing today where I asked a Sonnet 4.6 agent to code review a proposed design plan that was drafted by Opus 4.6 - I do this lately before I delved into the implementation. What it came back with was a verbose output suggesting that a particular function `newMoneyField` be renamed throughout the doc to a name it fabricated `newNumeyField`. And the thing was that the design document referenced the correct function name more than a few dozen times.

This was a first for me with Sonnet. It completely veered off the prompt it was given (review a design document) and instead come out with a verbose suggestion to do a mechanical search and replace to use this newly fabricated function name - that it event spelled incorrectly. I had to Google numey to make sure Sonnet wasn't outsmarting me.


Replies

rachel_rigyesterday at 1:53 PM

This is classic needle in haystack issue. Larger context windows make precision (especially with variable names) worse. Have you tried setting the context window smaller? 1M by default is a scam.