logoalt Hacker News

AmazingTurtletoday at 6:57 AM4 repliesview on HN

gpt-6-astra is a bitch, it constantly scope creeps itself with "yet another thing" to give it that darn polished lick. the results are eventually a little bit better but at what cost? let's do the math.

gpt-5.6-sol: 1x base gpt-6-astra 2.5x base in subscription

then gpt-6-astra tends to spawn subagents a lot, often with all kinds of models such as gpt-5.6, 5.3-codex etc., which is neat. it's a good coordinator but even more cost.

and then it tends to run _full test suites_ over an over again (each costs like 15 minutes) just to verify that _one test_ was fixed etc., and does so for as long as until the test is fixed, eventually accumulating 2 hours or so.

yesterday I assigned it a task to rebase my changs in a repo onto the latest upstream changes. while gpt-5.6-sol consistently took like an hour to do so end-to-end, astra ran for more than 6 hours and still wasn't done. it kept finding "one more thing" that was goldplating that I didn't ask for.


Replies

bob1029today at 7:04 AM

> each costs like 15 minutes

I've got a custom agent loop that will reuse unit testing results if no apply patch operations occurred since the last invoke.

Wall clock time isn't something I would put on the AI provider. That's entirely a consequence of the system that you've brought to the party.

show 1 reply
weird-eye-issuetoday at 7:01 AM

They don't always have a great concept of time so for something like running a full test suite that takes a long time you should just tell it not to do that

djmipstoday at 7:14 AM

have you ever worked for a big company where that's the status quo for any tiny change... hours on _full test suites_ over and over again.

show 1 reply