logoalt Hacker News

Supermanchotoday at 2:42 PM1 replyview on HN

> Efficiency, security, maintainability, reliability, readability, understandability, extensibility, maintainability, observability, portability

My experience is a little different. For higher abstraction languages the output is largely acceptable in my work. I always consider that LLMs don't know what I don't tell them and they have limited context to work from. Coding issues I often identify:

* Efficiency. Marginal by default. Coding efficiency problems often appear because LLMs dont usually consider the entire codebase or future plans (although they do guess at some futures). Sometimes they write/name things in ways that are lazy/wasted cycles. Most of the time, they don't.

* Security. Marginal by default. I say they do pretty good. Considering all the failure modes, not so much.

* Maintainability. Marginal by default. Mostly due to the careful consideration of modularity, upgrade paths, etc. while often taking wildly different approaches to solutions without having specific broad instructions. Even then, there can be big gaps in quality.

* Observability. Not acceptable by default. There's usually some consideration and can often one-shot.

* Portability. Not acceptable by default. Good, if you specify what those targets are. Regardless, testing validates this above the coding and models are very good at hitting functional test targets. This is less of an issue in something like Java ofc.


Replies

bla3today at 3:07 PM

This sounds roughly right to me, except for "maintainability". In my experience, agents really don't like deleting code unless you explicitly ask for it. If you're not careful, you end up with new better implementations of things but with the old implementation still around in perpetuity. Humans do this too of course.

show 3 replies