logoalt Hacker News

latentseatoday at 1:13 AM2 repliesview on HN

Those layers are deterministic though...


Replies

nomeltoday at 1:23 AM

Determinism is NOT a requirement, as proven by your existence of software devs. Again, shift the perspective to the people you work for, or to your fellow dev. They get along fine with you, who are also not deterministic. They check your output by setting requirements that you can prove are true in a way that satisfies them. You yourself guarantee correctness by tests, because you learned long ago that you yourself are not to be trusted, and are not capable of writing 100% correct code on first pass, except for the most trivial tasks. Your lack of determinism is built into your workflow, with tools like linters, code analysis, etc, to help deal with it all.

win311fwgtoday at 1:45 AM

Not really. It is true that supply chain attack concerns has put increased pressure on compiler makers to make compilers deterministic, but historically they haven't been, generally speaking. It was long thought that determinism didn't matter. Where compilers do differ from LLMs is that compiler output correctness is binary, while LLM output correctness is probabilistic.

However, compilers have long suffered from inputs not being correct, so we have put a lot of effort into finding ways to validate that the input is correct relative to the output. Which turns out works just as well for LLMs, to the point that validating that the input is correct relative to the output ends up offering the same assurances for both compilers and LLMs alike. Compiler output correctness alone is not actually all that beneficial in practice because, again, it is only as useful as the correctness of the input and inputs are not trustworthy without "double entry accounting" to go along with it. Once you have "double entry accounting" in place then you already have what you need to turn LLM correctness into being binary as well.