With every new model I've been attempting to build something that takes at least 1 month and really pushes the model. So far I've done:
- Python interpreter in Rust (fail): https://github.com/BlueBlazin/pyrs
- Low-poly Google Earth like engine (success): https://github.com/BlueBlazin/lowpolyearth
- Codebase -> 2d infinite-zoom visualization (success): https://github.com/BlueBlazin/blockviz
I have never understood or liked all the obsession with one-shot builds.
> I've been attempting to build something that ... pushes the model ... never understood ... one-shot builds
Same, and I have found some of the failure modes in the realm of eyes free coding and limited auditing. I often see it more as pushing the process than the model, as I tend to bounce around models for the benefits diversity provides.
Have you also found there is a point where they hit code complexity and start churning without progress? (at which point you have to step in to help the bot step back and think big picture, which usually takes a few sessions)
I used Fable 5.1 and Opus 5.5 to build a Python interpreter in Rust. Appears to pass all the 3.12 code I throw at it.
To get a toy version that can run most things takes under 2 hrs. Half of that time is collecting the tests you want to pass, how it should evaluate the code, what your criteria are, etc.