fwiw I have a bunch of LLMs writing first Lean code and now Agda code.

My observation. LLMs find reasoning about Agda as difficult as I find reasoning about C code. I've thrown a lot of gnarly C and Ruby code at all sorts of LLMs and they have only gotten more and more impressive as frontier models have gotten stronger. With Agda, they're like "hmm, tricky" whereas for me it's an impenetrable fortress. I've asked them why they find Agda so much more difficult to write (and why they have to iterate and reiterate many many many times until they get to a destination whereas they can one-shot and two-shot C and Ruby and they tell me its the multiple competing constraints. GLM is hilarious, it flat out refuses to write Agda code but it reads it well enough. They all read it well enough. Fable is obviously great at it. And Opus 4.8/5.0 are great (if they stay on track and don't sneakily go their own way) but they're too annoying to talk to. On balance Kimi K3 is the best balance of not annoying, relatively cheap, and strong -- great model all round tbh.

So yeah, interesting I've discovered the limits of their ability coding-ability-wise. None of them are that good at designing/aesthetic judgment/architecting so thankfully they still need me in the loop.