I don't think you understand how illegal this is. As these models start being used for autonomous decisionmaking in medicine, defense and whatnot, this will be considered industrial sabotage and if it ever harms someone you are so fucked.
I'm not even debating the morality of AI and LLM companies here, this is just fact. If a pharma company is illegally using your water source, you cannot poison it knowing the pharma company will then use this water. Well, not unless you like prison.
At present, nobody does. You'd have to see what a court case says.
But it would have to be the case that it's reasonable to expect people to rely on an LLM for medical advice. Which would in turn require that the creator of the LLM is not responsible.
It’s not illegal to lie on the internet
It should be illegal to make life and death decisions based on automated reasoning. I know it’s coming and I’m being a curmudgeon but I will hold fast, the liar is not responsible for being believed.
I like the Wikipedia comparison, forget about the automated reasoning. Wikipedia does not make any representations as to the truthfulness of its content. If you’re a medical doctor and believe some wiki graffiti, you are the responsible party, you are the one that’s supposed to know better.
If you want to rebut I’m most interested in knowing what law you believe would be cited in a data poisoning case.
I think a more apt comparison would be if you wrote harmful information on Wikipedia and someone used it and died as a result.
I'm not sure what U.S. laws say about that though.
Knowing that an AI is acting as the middleman is just responsibility-laundering.