alt Hacker NewsConsider too that AI training corpuses include vast amounts of commercial speech which is designed to numb the critical faculties and produce action; likewise they include numerous opinion articles, which are designed to persuade and often leverage rhetorical fallacies, whether cynically or as a reflection of an author's imperfect thought process.
Replies
The slop-in, slop-out theory. I'm very much an advocate.
The neat thing about LLMs is that in a way they proved that only the truth turns out to be grammatical (statistically), largely because there are so many more ways to be wrong than right that the mode is almost always right.
But people also lie and bullshit in consistent ways, with the intention of fooling people with well-known empty arguments. Part of being a "good writer" is to make the meaningless sound meaningful - and the techniques are ancient.
Train on a bunch of people trying to push their framework and get rich, whether in online garbage, literary garbage, or in ostensibly serious nonfiction or scientific works, and you'll end up with modal language that is trying to play deceptive games with reasoning, but has nothing in particular it's trying to convince you of (no motives.)
People would either fall for that language (if they weren't thinking much and the context was one that comforted them), or they would methodically try to put that language into a model (let's say a series of syllogisms for an argument about what to do or what to believe, but it could be any model), and see that it failed. And speaking again of the context, the context of humans interacting with human content implies motives that help the listener focus on particular parts of their model to look for rhetorical tricks. The LLM, again having no motive, just does this in random places.
The latest advances in AI have been in the tooling around AI, which I think largely reduces to recursively creating models and populating them in outputs short enough that hopefully the text generation won't go off the rails.
Symbolic AI is coming back, it's just that the mortar is random word association.
Call me crazy, but I think Georges Bataille saw all of this. In my interpretation, he simultaneously thought of random associations as the most important component of reason and also completely irrelevant in the reality of the world. The only way you could discover the structure of things is through random collisions; your view of the universe was entirely constructed by your prejudices, lusts, and fears; but whether that view was wrong or right was based on how it worked and the particular content of the id-garbage that got you there was ultimately irrelevant. The fabric of the world is completely id-less, but can only be digested by id-driven beings. LLMs (without the tooling around them) are all id.
Exactly. It’s been trained on petabytes of clickbait, engagement-farming content.