alt Hacker NewsMy expectation is that AI will give us a way to nicely quantify how productivity is impacted by choice of language. Because we can rerun the same request as often as we like and compare the results.
And I expect that it will turn out Python is the most productive. As it is most easy to reason about. It allows for the most elegant expression of the idea behind a program.
The first tests I have seen seem to confirm this. One recent example:
Replies
I think it's far from being this simple. What you're describing is productivity on a greenfield project, but what's really interesting is productivity when working on an existing large codebase, with existing conventions, architecture decisions (or lack of)... How easy is it to do a product pivot, to rearchitect for performance, etc etc etc.
It is unclear to me though how much of your expectation might be set by the training dataset.
For example, Python and Typescript have the most amount of codebases and training being done on. So I feel as if that plays a part into the overall thing.
Languages which are more niche have genuinely hard times (Try arturo lang for example), so it depends on a lot of things/nuance, or well that has been my experience trying something recently.
My personal opinion is that if each language has the same amount of training. Golang comes close but the first might be Elixir. I have seen Elixir language perform really well with LLM's with magnitudes less training dataset. There have been some studies which had Elixir as the number one language for such tests iirc.
Gleam is a new addition as well and I feel as if it could be good and its another interesting option as well with more type-safety and an interesting language overall.
How do you propose to quantify the terms "elegant" and "easy to reason about"? What unit of measurement do you use for these?
This sounds like your personal feelings, not quantification.
Purely anecdotal, but my experience has been that LLMs generate low quality python code. It's spaghetti code on par with what I've seen when companies I worked at tried offshoring development. It's basically what you get when you give bad or incomplete specs to a team of inexperienced programmers with poor development practices. It's usually good in small chunks, but it gets extremely sloppy as the scope of work increases and more decisions are introduced. Interestingly, I've seen LLMs generate good clojure code.
My guess is it comes down the the training data more than anything else, although I suspect functional languages will fare a little better. At least that's been my experience. There's undoubtedly a ton of python code in the training corpus and portions of it are of dubious quality. Niche functional languages likely have a smaller training corpus where a larger portion of it is better quality.