> even if the risk is smaller with the computer.

How do we even begin to establish that? This isn't a simple "more accidents" or "less accidents" question, its about the vague notion of "justice" which varies from person to person much less case to case.

But who controls the computer? It can’t be the government, because the government will sometimes be a litigant before the computer. It can’t be a software company, because that company may have its own agenda (and could itself be called to litigate before the computer - although maybe Judge Claude could let Judge Grok take over if Anthropic gets sued). And it can’t be nobody - does it own all its own hardware? If that hardware breaks down, who fixes it? In this paper, the researchers are trying to be as objective as possible in the search for truth. Who do you trust to do that when handed real power?

To be clear, federal judges do have their paychecks signed by the federal government, but they are lifetime appointees and their pay can never be withheld or reduced. You would need to design an equivalent system of independence.

I'd rather get judged by a human than by the financial interests of Sam Altman or whichever corporate borg gets the government contract for offering justice services.

> nonzero risk of unfair judgement from a computer

I feel like this is really poor take on what justice really is. The law itself can be unjust. Empowering a seemingly “unbiased” machine with biased data or even just assuming that justice can be obtained from a “justice machine” is deeply flawed.

Whether you like it or not, the law is about making a persuasive argument and is inherently subject our biases. It’s a human abstraction to allow for us to have some structure and rules in how we go about things. It’s not something that is inherently fair or just.

Also, I find the entire premise of this study ludicrous. The common law of the US is based on case law. The statement in the abstract that “Consistent with our prior work, we find that the LLM adheres to the legally correct outcome significantly more often than human judges. In fact, the LLM makes no errors at all,” is pretentious applesauce. It is offensive that this argument is being made seriously.

Multiple US legal doctrines now accepted and form the basis of how the Constitution is interpreted were just made up out of thin air which the LLMs are now consuming to form the basis of their decisions.