alt Hacker NewsI strongly believe that you cannot evaluate how good a system design is if you don't implement it by hand.
LLMs will implement what you ask them to, even if it is the wrong approach. They can be lazy and take shortcuts all the time, but they do not feel PAIN (obviously they don't feel anything and aren't lazy, I'm just personifying them but you get the point). Only when you implement by hand can you feel if the implementation of your design is painful or not, and only this signal can tell you if your design if truly good or not.
I do think LLMs are useful for design work, they are good at asking clarifications and probing questions which actually do push you to approach problems differently, but leaving implementation of designs to LLM is a recipe for disaster, and judging your own design skills when you're not implementing such designs is seriously laughable. And to be clear, it already was before LLMs, when "software architects" were just designing and then had peons implement for them.
LLMs are enabling a whole new level of bad code that is best describe by the following Jurassic Park quote: “Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should.”.
> A it reaffirms my belief that FAANG devs are fairly mediocre on the whole
Off-topic but having worked in other companies as well, I can guarantee you that this is not the case. The skill of engineers in FAANGs and other "top tier" companies is much higher than average.
Replies
> Only when you implement by hand can you feel if the implementation of your design is painful or not, and only this signal can tell you if your design if truly good or not.
I think this touches obliquely on a point I keep coming back to, that one of the most important things a codebase does is to communicate ideas about how a process should work. Yes, it also produces some binary that runs on a bunch of servers or whatever, but that's a really temporary, ephemeral artifact. The lasting thing is the idea. Making your ideas (expressed in code) easy to understand, easy to work with, and easy to evolve in time is the art of software engineering. I 100% agree, from my own experimentation with LLMs, glancing at something a model has produced and checking that it has some test coverage isn't enough to know whether it's well-engineered. You'll only find out later when you try to work with the code.
> I strongly believe that you cannot evaluate how good a system design is if you don't implement it by hand.
Fwiw we both agree that LLMs should not design systems. I do the design, but otherwise I don't get how this is true, the success of a design is indicated by long term success in the system it built. You can measure this against success in the task it was deployed for via performance metrics for one. And then from a developer standpoint how easy it was to maintain later on. Success of a system is a measurement over time, but it's not some quality that can only be measured by those who built it.
> Off-topic but having worked in other companies as well, I can guarantee you that this is not the case. The skill of engineers in FAANGs and other "top tier" companies is much higher than average.
I have first hand knowledge of this so I agree to disagree. Being surrounded by google, aws, and meta folks my understanding is the best people leave faang when they get the itch to do something better with their time.