alt Hacker NewsI liked the article. It was a long (and entertaining) build up to the conclusion, but I'm scratching my head how the author got there.
AI needs more discipline, yes. But theoretically that discipline can be learned much easier than becoming a good engineer.
Think of it this way... 20 years ago, to write good, scalable C code - you needed to 1) either be a genius, or 2) dedicated to the craft.
You need to learn dozens of tools like the back of your hand.
* ASan
* LSan
* UBSan
* TSan
* GDB
etc... God forbid if you needed to manually read DWARF files. Unless you're a pure genius, this is not feasible to master in a short amount of time. And in parallel, you need to learn how to design systems, too, otherwise, you're still not very good, and that's an almost completely orthogonal skillset.
Now, you simply need to be aware of the hazards in your language/framework, tell your LLM to test for them, have the infrastructure set up to see if they've adequately tested for those hazards, and maybe read the actual tests and implementation.
It is pretty easy to be able to read and understand Rust compared to debugging all the sorcery-like errors that come during Rust development... It is easy to see that you need a Loom test for certain scenarios, and to write a tool to detect if you did it.
Even if you're still working in C or Zig, it far easier to know and detect when you need to use those tools then to learn to use them all individually.
It is not hard to learn to read SQL. Almost ~50% of business professionals can. Python is barely harder. Rust can look like sorcery if you don't read a 50 page guide to understand to read it, but that's a VERY small price to pay compared to spending ~10 years learning the craft painfully by trial and error.
I'm not sure how you get from "LLMs work in mysterious ways" to "So we need more discipline" to "everything is fine."
I agree that everything is fine. I just don't think this is the clear path and thought process.
Anyone who has the determination to get things to actually work, and takes a little bit of time to understand what makes them not, should be able to leverage LLMs to work wonders.
In my opinion, LLMs are going to make things far more complicated, because the cost of building something complicated is becoming almost free.
Engineering was always about discipline and getting things to work. But you needed a set of prerequisite skills to have much value. Most of those are gone now.
It is simply far easier than before. It does require discipline, yes. But discipline is cheap compared to ~10 years of trial by fire.
> I'm not sure how you get from "LLMs work in mysterious ways" to "So we need more discipline" to "everything is fine."
Are you referring to this part:
> I am not worried, at least in the near term, about AI creating massive, discontinuous returns on investment in the absence of engineering discipline. (Many will try, and it will be entertaining to watch.)
She's saying, "the amazing thing about LLMs isn't that they generate lots of code fast, so don't worry about people using LLMs for that taking over the industry"
She's making two points:
1. Before infra-as-code, people would be afraid to touch parts of production due to lost knowledge about how and why it got that way. Now that we have infra-as-code, you aren't allowed to change infra the old way (ad-hoc changes via dashboard/CLI), even if doing so would be faster and easier. Experienced SREs were required to abandon lots of their old skills with CLIs and dashboards and start working in a completely new way, because the knowledge captured in a terraform repo's commit history is so valuable.
2. In the past, the way code got written was through people making changes in ways that are specific to their current knowledge, the org's current problems, the current users, etc, some or all of which is not written down. Eventually, everyone is afraid to change certain things because they don't know or remember all the considerations that went into them (not just afraid to touch parts of the code, but afraid to delete seemingly-unused features, or migrate the schema, or whatever).
Charity is saying that problem 2 is a hidden/lost-knowledge problem like problem 1, and the amazing thing about LLMs is you have to write down all the knowledge you want them to have, which may lead to a better solution to the "lost knowledge" problem in software development, which would be so valuable that experienced software engineers have to abandon lots of old skills and start using it.
(Not only writing down all the knowledge you want the LLM to have, since they're flaky enough to ignore instructions and miss implications sometimes, but building test suites and tools and so on that adequately guide their solution. This is the "more discipline" she's referring to.)