I still have to correct Claude on very basic misconceptions whenever I get it to code shit.
Sometimes it gets wrong things that I had spelled out already.
It may be the Doomsday machine, but it is a very silly one. If it kills humans it will do so by mistake.
"You are completely right! Humans cannot breathe sulfur dioxide! My mistake, and I take complete responsibility"
> I still have to correct Claude on very basic misconceptions whenever I get it to code shit.
Can you give a simple example?
I would have agreed 2 years ago, but it's extremely rare I see a frontier model making a silly mistake these days.
https://en.wikipedia.org/wiki/Instrumental_convergence#Paper...