logoalt Hacker News

embedding-shapetoday at 1:56 PM0 repliesview on HN

> Decisions only, never your messages

For me this is basically the opposite of how it would need to be, the decisions are all in my messages to the agent, almost never directly in the agent's replies, sometimes indirectly though I suppose.

Interesting to see that apparently some people use these tools and are the ones listening and doing what the LLM decide and says, rather than the opposite.

> Ran a blind eval before shipping, published it including the misses (REPORT.md). Agents with Grepathy answered the "why" questions right. Baseline agents made up confident wrong answers.

Is this eval public anywhere? The whole ecosystem severely lacks transparency, and people continue making strong claims without any sort of data or results that backs these up. Is there any public somewhere backing up the idea that it's more accurate that just using whatever agent harness?

> Claude made a decision nobody approved

It irks me a tiny bit that this is essentially clickbait, you don't know for sure Claude "made a decision nobody approved" as you've lost the data because you never thought sufficiently about storing these things in the first place. Most likely it misunderstood you at one point, you didn't carefully read the reply and approved it all, so most surely what you've done lead to that being done. These big models hardly ever just randomly do whatever, you can more often than not trace it to something you did wrong.