logoalt Hacker News

archargelodtoday at 9:01 AM0 repliesview on HN

You can't. You must prove it. And I don't mean that you need a rigorous scientific proof - that would obviously be too hard to do for every single function/library/program.

Human developer can work on a program incrementally, ensuring at each step that it is mostly correct.

But LLMs can't think, they fake reasoning and explore problem space in random walk until they stumble into something that looks like a solution. And these "solutions" will have hilarious and absolutely unexpected failure modes.