One of the biggest strengths of a computer is reproducibility. The worst software bugs are inconsistent or non reproducible. The least useful calculators apply rules inconsistently, to your example.

the inconsistency of LLMs is by far one of the biggest gripes I have with them. Closely related to their apparently deep desire to avoid following instructions.

I know these are both a byproduct of noise (which is somewhat tunable) and noise is inherent to these systems in a lode bearing way.

I still hate it. it’s holding the technology back. I don’t honestly see how we can safely or even successfully approach the idealized realm of AI without bypassing this problem, which to my understanding, probably means not using language models at all and trying a totally different approach. But I really don’t know much about machine learning, I’m a super novice compared to a lot on this website.