alt Hacker NewsTuning the model so far in the direction of being aggressively useful that it will quickly go off the rails in the name of helpfulness.
I swear I spend more time telling Claude not to do things than telling it what to do.
Replies
mdp2021 • today at 6:49 AM
> aggressively useful ... in the name of helpfulness
But is that because of training, or can that be (also? mostly?) an effect of the "system prompt"?
I guess the agentic coding benchmarks don't have many rewards for stopping and clarifying what the user wants?