> With compilers there is a (roughly, barring optimizations) 1:1 correspondence between input and output.

This is repeated in a lot of comments here but hasn't really been true for a while. I remember, c. 2010, Kenton Varda complaining to Google's internal compiler team about how it generated some particularly non-intuitive assembly code out of the protobuf generated code, and the response was "Yeah everything is adaptive optimization now. Benchmark it and give us a profile and we'll put it in the compiler test suite. Trying to reason about what the compiler is doing and micro-optimize accordingly hasn't worked since the mid-90s."

I do think that LLMs are a step change in how we program computers, but the main step change is that the source language is now English, which opens up programming to a wider market accordingly. And this'll come with all the downsides of English: the ambiguity, lack of precision, multiple meanings, and so on. If you try to recover that control over what the computer generates, you end up reinventing computer science. (Which isn't that far off from how I use LLMs to program: anything that has widely accepted "best practices", I'll just let the LLM do, but if there's anything novel or requiring high-precision, I'll tell it "leave that function body blank; here are the inputs and outputs to the rest of the program; I will fill in the algorithm details myself.")

I also think that people are underselling the extent to which compilers were a step change to previous practices of programming. For someone accustomed to C, try doing register allocation or inlining yourself; you will suck at it, and it will take up the vast majority of your head space that could be spent on better things. For someone accustomed to a garbage-collected language like Python, try doing manual memory allocation and object ownership yourself; you will suck at it, and it will take up the vast majority of your head space that could be spent on better things.