logoalt Hacker News

Someone1234 • today at 6:20 PM • 19 replies • view on HN

My big hope for this release is - Have they got their massive over-engineering under control?

Say what you will about Opus 5.5, but it has a good "gut instinct" for how large/complex to design the solution. The previous Astra just absolutely loved pointless abstractions, excessive numbers of files, and just over-complicating very simple things within an inch of their life.

And, yes, you can prompt your way out of anything. But needing a default prompt to tell OAIs models to work better isn't preferable, particularly when that sometimes gets lost in sub-agents and the over-engineering returns into parts of the work.


Replies

seizethecheese • today at 7:09 PM

I was skeptical Sol 6.1 actually over-engineers compared to Opus 5.5, as I just switched from one to the other and haven't noticed a difference, but it looks like you're probably right!

We're building a battle system for OpenCode (https://agentduel.com/leaderboard) and have data on how large the diff was compared to opponents for both:

Opus 5.5: 0.42x (based on 299 battles) Sol 6.1: 1.42x (based on 265 battles)

Data on this is still pretty small. P=0.14. It's also affected by large differences, this is average not median.

➕ show 3 replies
scandinavian • today at 6:31 PM

I've been using both sol and astra to write some python and rust, and they generate the sloppiest code I've seen in a long time. I would be embarrassed to commit anything they write. The rust code especially was terrible. Had it implement a reader/writer for a binary format and it was just littered with magic numbers, terribly named variables, very compact code visually etc. It produces working code, so I guess the GPT models are mostly for people who let AI take the wheel 100%.

➕ show 6 replies
pettijohn • today at 7:02 PM

Oh my god yes. 6.x has been terrible at overcomplicating things. I have to prompt it to find the simplest solution that meets the requirements and then ask it to simplify and tell it to STICK TO THE PLAN. In addition it feels like they overfit for persistence. I created a new work tree and forgot to copy .env files. I caught codex hammering away and docker and psql commands and stopped it and asked it to explain what it was doing. So now my agents.md says "if documented tooling fails STOP and seek human assistance."

➕ show 1 reply
buybackoff • today at 8:17 PM

I've been fighting with this Sol/Astra tendency. I noticed that Sol Light is less bad with that, the higher the effort the worse that bias to overengineer is. It takes time to understand the limits, buе a dumber model that is faster and does less annoying things was better for my productivity as default.

In addition, I have spent some time to fine tune my custom instructions for Codex. It went like every time it violated my desired behavior, I asked why it did not understood my instructions and did this and that. In the end, the resulting instructions seem to work.

In the process, the model explained that the master instructions built in the Codex just tell it to push to the end and do as much as possible to satisfy a user. And these master instructions have precedence, so it's better to put the custom instructions into the developer_instructions section in the toml config file, as it has more weight, and not (only) in AGENTS.md. I ended up having both, with more details in the system AGENTS.md.

➕ show 1 reply
afro88 • today at 7:00 PM

I have been bitten by this 3 times and each time Opus 5.5 brought it back to something sane. The last time I left Opus in a /goal to get behaviour matching the docs, and it decided on it's own that a rewrite was the best path (and it was, and it worked)

contextfree • today at 7:25 PM

Man, I haven't tried any Astra yet but I find Opus 5.5 rather bad about this already.

CSMastermind • today at 9:00 PM

This is 100% true by the way.

I actually have the problem with Fable stopping too early on tasks but Astra way overengineers things.

riazrizvi • today at 7:58 PM

I don't mind it too much. Refactoring the bloated poorly integrated code is one way I stop my skills atrophying.

It also reassures me about the longevity of my career. I have sustained a theory that there simply isn't the data to teach these things to be really good, so I love to get the evidence to back it up.

holzhaker1 • today at 6:27 PM

yeah i feel like with astra they went all in on 3D modelling which encourages "just one tweak, just one more tweak" which makes it detailed enough, but in coding that same behaviour just sucks.

➕ show 1 reply
geonic • today at 8:04 PM

Exactly my experience. Codex loves to generate unnecessary complex solutions. Having Fable level intelligence at 5% the cost is a very good reason to switch though.

➕ show 1 reply
pants2 • today at 6:35 PM

Lol I made the mistake of having Astra design and build a complex system without any input (experimentally). Seemed functional at first and shipped it internally and it had dozens of complex bugs, so I started digging into the architecture and it was a monstrosity! Definitely 10X more complex than it needed to be.

➕ show 2 replies
x3haloed • today at 8:32 PM

I miss 5.6-Sol’s attack dog nature. Everything in the 6 family just feels like it’s sleepy.

➕ show 1 reply
mpalczewski • today at 8:08 PM

opus 5.5 hallucinates all the time. and tends to be overly verbose. and no you can't prompt your way out of it. you need checks and verifications.

➕ show 1 reply
jeffybefffy519 • today at 7:49 PM

This is why i stopped using GPT 5.6 onwards... in my view GPT 5.3 Codex is when they peaked for coding.

incrudible • today at 6:45 PM

I have limited experience with OpenAI models, but I have to say that Opus 5.5 has not improved on the terrible writing style, and I am struggling to prompt my way out of it. It always needs an extra pass dedicated just to rewrite all the prose.

➕ show 2 replies
surgical_fire • today at 8:14 PM

> Say what you will about Opus 5.5, but it has a good "gut instinct" for how large/complex to design the solution.

I have to correct stuff Opus suggests during planning every single day, not even for major tasks. Very useful still, but last thing I would suggest to anyone is to trust its "gut instinct".

Small wonder why we have news of outages everyday now.

tamimio • today at 8:11 PM

Spot on, identical to my experience. Sol 6.1 was a total failure that I won’t even consider it a frontier. I tried it on few problems at xhigh, it kept going miles after miles around the issue, creating massive unnecessary code and files and documentations and tests, even tests for test too, burning ~2mil of tokens over a week, then I fed the same problems into opus 5.5, less than half an hour and solved it, pinpointing where the issue was, where sol was wrong, and far cleaner code and output.

risyachka • today at 8:20 PM

>>> yes, you can prompt your way out of anything.

you really can't. unless you are working on something simple or what has been solved 100 times before

uw_rob • today at 6:31 PM

Honestly I don't even know if you can prompt your way out of these types of behaviors - stylistic tics and judgement seem to be extremely difficult for the models to overcome. It seems we are tied to what the labs do unless we want to dive into and get dirty with models yourself. In retrospect with time it becomes obvious that the previous generation of models couldn't be prompt engineered to do what the current generation of models do. There's just no way you will prompt engineer Opus 4.8 to write as well as Opus 5.5. I'm guessing with time we'll see that there was no way you could prompt engineer Sol 5.1 to be as token efficient as Sol 5.2

➕ show 1 reply