I've been thinking a fair bit about what I'm seeing in terms of the output I experience
It's quite hard to quantify, but I think it's one shot nature really makes it hard to gauge it's capability
Friends have spoken of good days and bad coding days with me, and I find it odd nodding along, it's a strange new normal
At times it feels like we're just coding with one-armed bandits, trying to carefully line them up for a jackpot and just discarding and retrying if we don't hit
I think about some of the more complex systems I've built and I wonder how well we can build them like this
And over engineering, there seems to be over engineering everywhere, and yet, more fragility to our systems
It's all a little surreal
I imagine this stuff is probably really good at iterative changes to improve objective benchmarks like CPU or RAM use. I'm thinking of little contained optimizations that you can understand in and of themselves, that you maybe have done yourself before in other places, found really quickly and applied uniformly. Stuff you can confirm to be what you expect it to be by mostly just scanning. Low hanging fruit for sure, but something where you can actually know what is a fruit and what is crap, and only keep the fruit, and develop some confidence in the process, if that makes sense?
Or trying to reduce complexity, increasing readability and coherence of variable names (the opposite of code golf if you will), while staying within a certain limit of performance regression (e.g. "make this code as nice as you can while making it at most 0.5% slower").
Making the stuff millions of people have been using for decades better, in a way that also makes it better for humans when they read the code. Surely that's possible, some people are probably doing it but it doesn't go viral as much, because it's too mundane.
And of course, making new stuff is more exciting. I mean, you could hit on something with a vibe coded thing, and then know it's now worth to make a non-sloppy version, but you won't get much fame for making ffmpeg twice as fast by prompting an LLM. Though on the other hand, it's like a safe investment (if not in "fame", then in "improving the stuff we all have to use daily"), because you know ffmpeg and many many other things will still be around, whereas a vibe coded thing that wasn't special will be 100% forgotten the next day, or have just the one user forever.