alt Hacker News> I was surprised by my own ignorance and how much LLMs like to complicate stuff.
Exactly and it's like a plague. I think the main reason is that LLMs are generators, they like to generate code where code removal or reduction would solve the problem just as well. You ask them to fix something, they generate a fix. You tell them the fix didn't work, now they generate a fix for a fix, whereas a fix that would also reduce complexity existed from the start.
With a fair amount of babysitting and constantly reminding the agent that code removal is an option, you can eventually get a decent codebase but I wonder why this isn't a default. Because they are trained on mediocre code that is the vast majority of what's available on the internet? Most likely.
> the main reason is that LLMs are generators, they like to generate code where code removal or reduction would solve the problem
Don't over-index on this choice of words. LLMs can just as easily generate a diff that removes or refactors code, they don't have any intrinsic "preference" to always add more code, unless it's been trained into them.
A lot of the early training had concentrated on "wow factor" one-shot style demoable use cases, where the models indeed ended up with a tendency to assume they were in a "produce something new for me right now" situation and often or always started churning out additional code, even without all relevant information. As training regimes improve, and as customer demand evolves, this is changing, and the latest big models place a lot more emphasis on maintenance tasks - still not enough, but it's at least in the mix. As they mature, I'd expect the models' "instinct" to gradually align with what everyone's idea of a senior software engineer is, taking time to figure out if they can simplify and refactor rather than always adding more.
It's a new field, it will take time to get it right, but I don't doubt it will eventually be right.
> Because they are trained on mediocre code that is the vast majority of what's available on the internet
Again, this is outdated. It's true that the earliest training took this "spray and pray" approach, but data quality filtering is now a huge focus and the labs have made a lot of effort (and progress) in figuring out what good code is and getting the models to follow that. They've massively improved, in my experience, and it'll only get better. Again - give it time, this is new.