I’ve had a similar experience using LLMs to have mini personal breakthroughs.

One thing I notice is many models say statements along the lines of “okay we have exhausted this thread it’s diminishing returns from here and we should stop and move on”

It’s funny because I’ve been building a tiny neural network maze solver (23 bytes solves 92.75% of unseen 2D mazes)

When I asked ChatGPT/Fable if we had anymore threads to pull to increase capability and decrease byte size, they both basically said no way - back when I was at ~166 byte models with a ~85% solve rate.

Throughout the experiment I just kept trying different approaches and eventually had 3 mini “breakthroughs” in this particular niche. But if I had listened to the models…

Anyway, these models are amazing to experiment with quickly, but they are dumb as hell and so absolute