alt Hacker NewsI think most of this is correct, in spite of potentially being built on a bad assumption.
The assumption is that LLMs should be writing the code and human engineers reviewing and verifying the LLM output. And that this pushes the cost of producing down. And I fundamentally disagree with that.
Every time I ask LLMs to write code, even with Opus 4.8 (haven't tried it with Opus 5 yet), what I get ends up being totally rewritten. LLMs still aren't good at writing maintainable code. Can they write plausibly functional code? Yes. But it won't survive the long term. People using LLMs to write all their code are gambling on them eventually getting to a point where the LLMs can fix their own code. It's possible, but I wouldn't necessarily bet on it.
Where I have found immense value from LLMs is in code review. Repeated review by LLMs catches an amazing amount of potential issues. They really shine on security review, but are very effective with any kind of review.
The other thing that the "LLMs write code camp" misunderstands is that writing was never the bottleneck. Understanding was. And understanding the code is still the bottleneck. But understanding is truly gained during the writing loop. The understanding you gain from pure reading or code review is marginal compared to the understanding you gain while writing.
Most of the time previously spent writing was actually spent updating and deepening our understanding of the system under development. There's no replacement for that understanding in a world where LLMs are doing the writing.
But if you flip it: humans write, LLMs review, then you still get a major gain -- not in speed, but in quality. And you keep the understanding loop intact. I would propose that this might be the best way to deploy LLMs.
Replies
I write code in Go in an established codebase. 90% of the time the LLM gets a basic api/feature right and fully e2e tested as long as I give it enough business context in the prompt. The other 10% of the time I have to do some follow up prompts to either change the behaviour or change the approach of a given step in the flow or just to point out that the wrong pattern was used and please rather use our codebase standard. I haven’t actually hand-written code in well over a year. Is this way faster? Absolutely. Does it lead to better code? Yes, because I have time to write all the regression tests that keep the behaviour as I wanted it when others come bungling around.
" But understanding is truly gained during the writing loop. The understanding you gain from pure reading or code review is marginal compared to the understanding you gain while writing."
Debugging code step by step is how I understand complicated code.
You're right about understanding. Where we might disagree is the conclusion. I think it's possible to build the understanding without having to type out syntax by hand (debugging, writing tests, etc). Maybe we're missing much better verification tools? LLMs will likely play a big part in those too (explain the codebase, walk me through A, B, C etc)
I've come to realize that the smarter LLMs get, the less they understand the point of abstractions.
I've been using it for a unity game for the past few years. Nowadays it will go sleuthing into packages and assembly and make decisions based upon what it sees there.
It will make comments about why it's doing something based upon a function call 3 methods deep.
God forbid any of these details change in a minor version update.
many of us work with swarms of cheap off-shore contractors and have to rewrite or finish their code already as it is. The llm is much lower friction
How do you handle the fact that an LLM can't seem to help itself from disgorging page after page of words no matter what it's asked to do? I've never seen an LLM say "this looks good as-is; I would not spend any more time on it; what's next?" it will always seem to suggest using another pattern or additional abstractions or other yak shaving.
But to be fair human code reviews have the same problem. It's like reviewers feel they have not done their job if they don't find something wrong.