logoalt Hacker News

a_t48 • today at 9:00 PM • 1 reply • view on HN

Against better judgement, I ran it on my repo. Poking at the "Ask anything about the report" bot, it penalizes me heavily for not having "positive tracked outcomes", as I don't report back to Claude what the outcome is. It wanted me to do this:

"At the end of research-heavy sessions, add a tiny closure note:

Decision: Use CacheMountStore with registry/local/GHA backends.

Why: GHA cannot expose the same content.Ingester path, local import has discovery issues, registry can resolve by tag.

Proof / current artifact: See files X, Y, Z. Subagent found A, B, C.

Next action: Implement interface in package N. Do not add new cache-mount flags yet. Use mode=cache-mount on existing cache flags."

There's no need for this, Claude is not my task tracker.

It also penalized me for:

    - Using Claude to introspect a codebase as throwaway work and ` not close with a crisp “acceptable / risky / copy this / avoid this” decision this session.`. No action was needed, it wasn't an actionable session!

    - Using Fable for code review. `Several BuildKit subagent sessions produced research reports, but the transcript does not show whether those findings became implementation choices.` Yes, Fable launches subagents to code review. Valid findings get turned into fixes or WONTFIX.

    - `The Cloudflare CI upload failure got narrowed well, especially after you supplied the exact timestamp and challenged the /v2 routing recommendation, but the session ended without a chosen next diagnostic owner or action this session.` There's no need to pick a next diagnostic owner when I'm a solo founder with no employees.
I'm choosing not to attach a report to my application. As a side note, nobody needs to see "my greatest crashout", it's embarrassing enough for me to see it for myself, much less show others.

Replies

drawnwren • today at 9:37 PM

I had the exact same experience. Claude is not my validation engine, I do not tell it when it has completed. I verify that in the code/project and end the session.

I also got this,

> A useful next habit is to end each correction with a concrete acceptance test, owner artifact, or stop condition: “write it into PROGRESS.md,” “make nix run .#bench fail until this is real,” “rerun this exact command,” or “do not proceed until these two choices are explicit.”

> You already do this well in the biggest penance sessions. Apply it to the smaller ones too.

Which I have found to be counterproductive in my personal work. Current models can generally infer acceptance tests of this level of granularity (not true for larger project-level prompts, but those don't produce good enough code for me yet -- even with specific acceptance criteria).

I also got penalized for using claude in read-only mode for the same validation reason?

> For read-only work, end with one of:

    “turn the top finding into a PR-sized plan”
    “mark these as accepted/rejected/deferred”
    “write a cleanup checklist”
    “give me the exact command I should run safely”
    “stop, no action recommended”
No thanks, I'm literally just exploring the codebase. I don't want any of these.

It's a little sad to be honest, I would actually enjoy a product that helped me improve prompting + ai usage.

➕ show 1 reply