The thing that makes it work really well is to make sure it has all the tooling to verify its hypotheses. If you allow it to run the full lifecycle in loops you will be surprised how well it works.
Exactly. In the above case, AI should be able to verify it's own hypothesis by having access to the database's table statistics and planner.
That’s how I do it in my Laravel apps. I simply ask it to hypothesize, benchmark, apply and verify. Laravel has great tools for that.
But them it's more like AI assisted performance fuzzing? Just throw stuff against some measuring harness and see what sticks?
Built exactly this for work a few months ago. Claude has a read-only account to the dev DB and a bunch of python postgres tools, I point it at queries/views and ask it optimize, and it has to:
1) verify identical query results
2) run repeatedly to get average, worst, best, etc duration of runs
Sped up so many legacy things that none of us were ever going to bother with.