For the problems I work on with GPT 5.6 Sol and the checks and balances I have in place, I estimate:
- 80% of prompts get everything correct and are confirmed correct with manual validation
- 19% of prompts make a minor mistake based on an ambiguity of the original prompt (user error not LLM error), but then reliably fixed in a followup prompt
- 1% of prompts causes more problems than it solves and is more pragmatic to just revert
For 99% good output, there isn't much of a dopamine rush when there is good output. The dopamine rushes are for the <1% odds.
From the other replies on this post, I suspect no one believes me, but I am offering these numbers in good faith.
alt Hacker News
For the problems I work on with GPT 5.6 Sol and the checks and balances I have in place, I estimate:
- 80% of prompts get everything correct and are confirmed correct with manual validation
- 19% of prompts make a minor mistake based on an ambiguity of the original prompt (user error not LLM error), but then reliably fixed in a followup prompt
- 1% of prompts causes more problems than it solves and is more pragmatic to just revert
For 99% good output, there isn't much of a dopamine rush when there is good output. The dopamine rushes are for the <1% odds.
From the other replies on this post, I suspect no one believes me, but I am offering these numbers in good faith.