alt Hacker News> Lead with conclusion.
I would presume (perhaps falsely?) that an instruction like this would lead to the model presenting a conclusion not supported by the evidence, and potentially backtracking as it then tries to justify said conclusion.
Yes, if deliberation happens, the model should figure out what it wants to say during that phase; but if you're using auto mode, the model is not going to be doing any deliberating half the time. In those cases, the output blathering is the model's only chance for deliberation. It "thinks as it talks", per se.
Given that, I would advise a different approach: let it blather, but then get it to write you a conclusion at the end that the model can guarantee will obviate the need to read any of the blathering.
I.e. advise the model to add an "executive summary" to the end of any non-trivial-in-length response. With some wording to carefully navigate the model between "the summary is itself too long" vs "the summary acts more like clickbait, leaving out necessary detail such that it requires actually reading the blather."
Not sure exactly what that wording would look like. I imagine something like "write your postscript executive summary as if you were a senior CIA intelligence analyst summarizing ground-level reports into a daily digest for the Joint Chiefs of Staff. Take up as little of their time as possible, but ensure that any detail critical to decision-making is retained." (But that phrasing might only be useful if the model is delivering a certain type of response, and actively counter-productive otherwise. This kind of thing is delicate.)
Replies
This was a big concern for earlier models, but with modern CoT trained models they should be able to come to the conclusion entirely in the thinking trace.
Oh the number of time LLM will, for example, be giving me the list of bugs it found in code, when I ask it for a review, just to decide there’s no big half way through explaining it.
You are absolutely correct. The second suboptimal part of the prompt is this:
> Trim introductions, repetition, generic reassurance, and optional background first.
It's not possible for the model to "trim" those before they've been output, so this is akin to telling it "not think of an elephant or even take the existence of elephants into consideration while solving this problem".
Yes this is an extremely well known result for exactly the reason you guessed. It's not just abcktracking, asking an LLM to present a conclusion and then justify is also an excellent way to provoke hallucination as the model con concts "any justification that plausibly justifies the words it's already said".
This is the actual reason why openai _invented_ reasoning models, to give them time/space to work out a solution, rather than having to magic a correct solution out of thin air from token 1.
It's less important now that all models do reasoning, but it's still almost always better to make the output come out last rather than first.
I wonder if it would help to ask it to write a rough draft and then reorder it?
I don't expect that would be the case. This is what's called BLUF or Bottom Line Up Front: https://en.wikipedia.org/wiki/BLUF_(communication)
The model will still have read the entirety of the document before composing its response. And I believe that even in auto mode, there are thinking tokens behind the scenes.
Over hours of experimentation with various LLMs, I've found virtually any system prompt can cause unintended skewing of the model's output. Even just 5 to 8 short, direct words about length, tone or formatting can cause subtle yet significant changes in model output.
Longer, more detailed or conditional prompts always introduce an additional cognitive load as it checks every token it generates against the conditions. Making instructions more absolute (like: "Never do...") can increase the duration of compliance but at the cost of creating a significant center of attentional gravity. This can cause far more output distortion as the model devotes increasing portions of its attention budget to ensure compliance with a heavyweight requirement or prohibition. Every word in a global prompt is a trade-off between attention, compliance, drift, etc.
As someone used to thinking of computers as natural deterministic rule-followers, it's weird having to carefully wordsmith and A/B test even the simplest global prompts. It feels like coaxing a hyper-literal, emotionally sensitive, spectrum-ish toddler to comply but without being so strict it gets 'upset' or spirals into hyper-focusing.