I've worked with these systems for four years now and they have not meaningfully improved in that time frame.

We still have:

- statistical correlation between two things will always cause one thing to lead to the other, no matter how much you prompt it to not have that connection (to be expected with a stochastic system)

- Math completely fails in longer contexts

- "thinking" token generation being on the correct track just to 'no, wait' on an already correct conclusion

- smearing of properties between logically distinct objects (a red ball and a green cube can quickly become a red cube and a green ball)

It could be that the set of your day-to-day workload which could feasibly be accelerated by AI just happens to be saturated around Opus4.5, but you can still see lots of “reasoning” which makes you think the model is more performant in the first days of use. That’d mean you couldn’t perceive any meaningful difference in more powerful models’ results, even though you can see a difference in the raw output due to the length of reasoning traces leading up to the result.

So for example, if your workload was literally just addition of sets of numbers, you’d never have noticed progress in the result beyond GPT3.x level models. But you would perceive a difference in the now-Tolstoyan length reasoning text accompanying the result.

5 seems incredibly smart to me in my conversations today about some pretty niche ideas in.longitudinal modeling. It.felt.like a big step up.from 4.8, to me

It's called frog boiling.

We get used to the new level of intelligence so fast, any deviation feels like going back to the stone age.

If you don't believe me, create something complex with Opus 5 and then with Opus 4.5, and notice the difference.

Going to call it user error if you find Opus 4.5 better than 5, sorry.

Well, what kinds of things do you see Opus 4.5 completely fail at? Maybe those are not the ones that newer models have improved on.

I honestly just use GPT models nowadays, Claude models are too restrictive and more of a quitter and fable/whatever is just too expensive to be worth it.

Enshittification.