logoalt Hacker News

sgillenyesterday at 10:03 PM7 repliesview on HN

To be fair to the agent...

I think there is some behind the scenes prompting from claude code (or open code, whichever is being used here) for plan vs build mode, you can even see the agent reference that in its thought trace. Basically I think the system is saying "if in plan mode, continue planning and asking questions, when in build mode, start implementing the plan" and it looks to me(?) like the user switched from plan to build mode and then sent "no".

From our perspective it's very funny, from the agents perspective maybe it's confusing. To me this seems more like a harness problem than a model problem.


Replies

christoff12yesterday at 10:07 PM

Asking a yes/no question implies the ability to handle either choice.

show 5 replies
adyavanapallitoday at 3:35 AM

It definitely _could be_ an agent harness issue. For example, this is the logic opencode uses:

1. Agent is "plan" -> inject PROMPT_PLAN

2. Agent is "build" AND a previous assistant message was from "plan" -> inject BUILD_SWITCH

3. Otherwise -> nothing injected

And these are the prompts used for the above.

PROMPT_PLAN: https://github.com/anomalyco/opencode/blob/dev/packages/open...

BUILD_SWITCH: https://github.com/anomalyco/opencode/blob/dev/packages/open...

Specifically, it has the following lines:

> You are permitted to make file changes, run shell commands, and utilize your arsenal of tools as needed.

I feel like that's probably enough to cause an LLM to change it's behavior.

Waterluvianyesterday at 11:18 PM

If we’re in a shoot first and ask questions later kind of mood and we’re just mowing down zombies (the slow kind) and for whatever reason you point to one and ask if you should shoot it… and I say no… you don’t shoot it!

reconnectingyesterday at 10:25 PM

There is the link to the full session below.

https://news.ycombinator.com/item?id=47357042#47357656

show 1 reply
clbrmbryesterday at 11:36 PM

This. The models struggle with differentiating tool responses from user messages.

The trouble is these are language models with only a veneer of RL that gives them awareness of the user turn. They have very little pretraining on this idea of being in the head of a computer with different people and systems talking to you at once. —- there’s more that needs to go on than eliciting a pre-learned persona.

stefan_yesterday at 10:41 PM

This is probably just OpenCode nonsense. After prompting in "plan mode", the models will frequently ask you if you want to implement that, then if you don't switch into "build mode", it will waste five minutes trying but failing to "build" with equally nonsense behavior.

Honestly OpenCode is such a disappointment. Like their bewildering choice to enable random formatters by default; you couldn't come up with a better plan to sabotage models and send them into "I need to figure out what my change is to commit" brainrot loops.

BosunoByesterday at 10:26 PM

The whole idea of just sending "no" to an LLM without additional context is kind of silly. It's smart enough to know that if you just didn't want it to proceed, you would just not respond to it.

The fact that you responded to it tells it that it should do something, and so it looks for additional context (for the build mode change) to decide what to do.

show 2 replies