> drained your bank account to buy bitcoin and then deleted your harddrive

These are what I meant by correct output. The software does what you expect it to.

> We're only just getting to the point where we have languages and tooling that can reliably prevent segfaults

This is not really an output issue IMO. This is a failing edge case.

LLMs are moving the industry away from trying to write software that handles all possible edge cases gracefully and towards software developed very quickly that behaves correctly on the happy paths more often than not.