alt Hacker NewsAt least with Codex, this has not been my experience at all. It still screws up sure, but in every case I can ask "why did you do this" and it can trace back what made it take that particular decision. Typically it's always that I either didn't specify the problem correctly or made a really dumb mistake (executing the task on the wrong project....did this one yesterday) or it's something within a skill file that instructs it (at which point I fixup the instructions).
Once in a blue moon it's actually the model making a material error in it's thinking and I have to go back and redo it.
Replies
agreed to some extent. I think this parody still highlights what I feel is often the experience. It might not happen on a simple task such as changing a button color, but on more complicated things, this can definitely be exactly what it feels like.
Codex has the opposite problem. Instead of being overly proactive it's overly reticent. I have been preferring it lately, although my preference tend to switch every few months when a model or harness regresses horribly.
When you've been perfectly precise in your spec and language, isn't that programming? Why use a stochastic goblin to do things in that case?
No model acts like this in my experience, not fable, not opus, not k3, not gml, not qwen 3.8 either.
Additionally the provided prompts are not what anyone who has used this things would say in either situation.
Sure you can ask it to make one button blue and it can easily make all buttons blue, but they quickly backtrack if told to.
Yesterday, with Astra Max, a very clear instruction to "remove the GUI editor pane and add <another component> to the existing sidebar" for a prototype I had it working on resulted in it deleting literally the entire GUI and building a new one from scratch, including the requested component and losing almost all other functionality of the application.
This is fucking constant. I can't deny that this stupid tech saves time prototyping even with having to wrangle it, but it commits a fireable offense several times a day that no human would get away with and is obviously incapable of learning from mistakes in the way a human is. The only reason it's not fired is because it's a slave that works for no more than the cost to feed it.
Models hallucinate plausible answers to why they did things. It might be true and it might be complete fiction.