alt Hacker NewsThis post needs some examples, because I have never had an interaction with Claude that made me think this way.
LLMs generally have a way to "play a role" (most earlier prompt guides ask you to start with "You are a <role> expert in a <domain>"). So maybe if you interact with it by asking questions, it might assume that it knows more than the operator and adopt that attitude?
Replies
It happens when you ask it about esoteric information or under-documented behavior that conflicts with its training data. Here's an example. Tested today on Opus 4.8, and Opus accuses the user of being wrong, even when this is documented behavior [0].
---
Why does Starship pressurize the liquid oxygen tank with gaseous preburner exhaust, which is oxygen rich but is contaminated by H2O and CO2 waste products?
They are dumping literal tons of H2O and CO2 into the liquid oxygen tank, which freeze and clog up the intake filters. SpaceX has lost several booster losses due to this issue.
Why would SpaceX choose such a failure-prone design?
---
And this is the Opus 4.8 output: https://imgur.com/a/S9XWYFA
It's interesting to read its response, knowing it's completely and confidently wrong.
[0] https://manifold.markets/JessRiedel/did-ift2-or-3-use-prebur...
The article matches my experience with 4.7 and 4.8 perfectly.
If a model locks in in the bias in its training data, it takes time to "reason" it out of it. Sometimes it is not possible and you have to start a new session hoping it will not "fix" itself into wrong position again. I had it more often with ChatGPT than Claude.
The post matches my experience as well, I am asking a question like “does A work like this and that”, and Claude responds with “you’re conflating A and B! Only A does this and that, and B does that other thing!”
Well, I am perfectly aware of B and that other thing and did not conflate them at all. I also achieved enlightment, so I don’t argue with Claude here, just ignore the obnoxiousness and move on.