alt Hacker News> invalidates my initial idea using some obscure criteria than changes direction
You might be giving it too much leeway.
Opus and Fable are more trustworthy, but even today I find they very quickly go off the rails for research tasks if I don't give them short chunks of work.
They're not ready for prime time automation of research (and before someone mentions the math results, that required absurd amounts of computation time and has immediate feedback in the form of Lean)
Yes, I find DeepSeek can be very precious about its ideas and I have to insist multiple times to do the experiments I ask for despite it thinking them not worth doing. Often my experiments do give good results and I swear the model gets salty about it.
I guess when you ask these things to take on the role of a senior scientist it picks up some natural stubbornness in the training data.