alt Hacker NewsTo me the difference is humans (ideally) will learn when they build something in a non-optimal way, and so will improve over time to become a competent engineer / architect. We cannot be perfect but to me a huge part of life is learning from failure and improving yourself, something that LLMs short-circuit and cannot replace.
LLMs cannot truly learn and so are destined to produce whatever the "average" software looked like at their training cutoff, or worse to produce code based on _other_ LLM generated code.
Ouroboros eat your heart out
Replies
throwthrowuknow • today at 12:52 PM
While I mostly agree, I think this is something we need to assume the Pareto principle applies to: likely 20% of humans will improve but 80% will not.
LLMs learn, and in two main ways: in-context and in training stages, release to release. The former is quick and sample efficient - perfect for adjusting AI behavior on the fly, and for enabling AI's own problem-solving capabilities. The latter modifies the "behavior defaults" and gives you performance gains that stick.
Why do you think that "write maintainable code" is somehow impossible to learn for an AI? We already have AI storming the frontiers of research math - way beyond the "average" of the field. If you can RL for "better at math", I see no reason why "better at maintaining code" would be somehow impossible.
You can construct an RL env where a codebase is presented as a "tree", and the AI is given one change to make at a time - and the per-change reward is not just whether the change itself has been evaluated as "made successfully", but also whether it made future changes down the line more or less likely to be successful, and harder or easier to make.
This is a formulation already used by some "maintainable code" benchmarks, so I expect something like it to make is way into frontier lab RL pipelines some time between "next week" and "a couple months ago".