alt Hacker News> but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading.
Hard disagree. LLMs (especially base ones, that only received pre-training) can produce output that is undistinguishable from human writing (because that's what they were trained to do).
But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either. And I don't think that's going to change anytime soon, unless their incentives change.
(We can say exactly the same thing about man-made stuff optimized for a specific purpose, like stock photography, clickbait titles or industrial food: they aren't stereotypical because their creator lacks the skill to make them otherwise, they are like that because that's what works best).
Replies
They're also designed to not offend anybody, so their output tends to be very bland even compared to the most milquetoast of human beings. I was only surprised once when ChatGPT responded with an enthusiastic "hell yes" seemingly organically, but 99.9% of the time these AI services clearly are instructed and trained to provide flavorless word vomit. I don't think there's a technical reason why an LLM couldn't produce totally convincing output, but internet grifters don't need to go through that trouble. It's like how most phone, email, and social media scams come off as completely transparent to most of us, but that's the whole point; we're not the target audience of the scams. Readers looking for substance, nuance, and real opinions aren't going to notice if something with written by an LLM – unless there are some cliche punctuation tells.
> But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either.
There are two problems with this.
The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maximize engagement".
And the second is that then people would just make other models that are tuned for defeating that sort of classifier, which would be used whenever the classifier is being used.
It does mean that this will have a drift problem if it's just trained on the idiosyncrasies of model fine tuning. That's fine! But it is something to be aware of.
> especially base ones
Did you actually try them? I did.They generated even more "slopey" text than instruction-tuned ones.