alt Hacker NewsLLM Writing Tropes.md
Comments
I work on research studying LLM writing styles, so I am going to have to steal this. I've seen plenty of lists of LLM style features, but this is the first one I noticed that mentions "tapestry", which we found is GPT-4o's second-most-overused word (after "camaraderie", for some reason).[1] We used a set of grammatical features in our initial style comparisons (like present participles, which GPT-4o loved so much that they were a pretty accurate classifier on their own), but it shouldn't be too hard to pattern-match some of these other features and quantify them.
If anyone who works on LLMs is reading, a question: When we've tried base models (no instruction tuning/RLHF, just text completion), they show far fewer stylistic anomalies like this. So it's not that the training data is weird. It's something in instruction-tuning that's doing it. Do you ask the human raters to evaluate style? Is there a rubric? Why is the instruction tuning pushing such a noticeable style shift?
[1] https://www.pnas.org/doi/10.1073/pnas.2422455122, preprint at https://arxiv.org/abs/2410.16107. Working on extending this to more recent models and other grammatical features now
If this bugs you, open chatGPT personality settings, choose “efficient” base style, and turn off the enthusiasm and warmth sliders
It makes a tremendous difference. Almost everything on this list is the emotional fluff ChatGPT injects to simulate a personality.
No mention of Claude/ChatGPT's favourite new word genuine and friends? They also like using real and honest when giving advice. Far as I can tell this is a new-ish change.
> Honestly? We should address X first. It's a genuine issue and we've found a real bug here.
Honorable mention: "no <thing you told me not to do>". I guess this helps reassure adherence to the prompt? I see that one all the time in vibe coded PRs.
A substantive human-written resource: https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing
As the article points out at the end, these aren't bad per se. The issue is that LLMs overuse them, and we're all getting the same(-ish) LLM. It's not so different from how people sometimes have their idiosyncratic phrasings they use all the time.
The article has been slashdotted so I don't know if this one is in there but:
One I've seen Gemini using a lot is the "I'll shoot straight with you" preamble (or similar phrasing), when it's about to tell me it can't answer the question.
Weirdly, LLMs seem to break with these instructions. They simply ignore them, almost as if the pretraining/RL weights are so heavy, no amount of system prompting can override it
What we really need is a browser plugin underlining these patterns, especially for comments.
I sort of think the whole middlebrow angst thread about Bourdieu going on right now applies to LLM writing
"The "It's not X -- it's Y" pattern, often with an em dash. The single most commonly identified AI writing tell. Man I f*cking hate it. AI uses this to create false profundity by framing everything as a surprising reframe. One in a piece can be effective; ten in a blog post is a genuine insult to the reader. Before LLMs, people simply did not write like this at scale."
This one hit home... the first time I ever saw Claude do it I really liked it. It's amazing how quickly it became the #1 most aggravating thing it does just through sheer overuse. And of course now it's rampant in writing everywhere.
This makes me think of the attractiveness of overly bad writing to writers, as a challenge, the most obvious example being the bulwer-lytton award, or the instinctive ignoring of instructions from fiction magazines that might say "we don't want any stories about murderous grandparents, French bashing, bestiality, bank robbers from the future, or kind-hearted Nazis - and especially do not try to be super brilliant and funny and send us your story about kind-hearted Nazi bank-robbing french-bashing grandparents that like killing people and having sexy fun times with barnyard animals! Because every original thinker like you thinks they are the first to have come up with that idea!" and then as a writer you feel challenged to do exactly what they say they don't want because what a glorious triumph if you manage to outdo everyone and get your dreck published because it's dreck that is so bad it's good!
It does not seem like there are lots of people who are perversely inclined to write a story with all these tropes and words in it, but surely there must be some, because if you make something that beats the LLM (by being creatively good) using all the crap the LLM uses, it would seem some sort of John Henry triumph (discounting the final end of John Henry of course, which is a real downer)
If only we could fix how it writes like garbage
Another trope: longer README.md's than anyone would make, or want.
Don't forget "The Ludlum Delusion"- every header in an article or readme reads like a Robert Ludlum novel title, ie "The [Noun:0.9|Adjective:0.1] [Noun]".
This list reads like, "AIs are not your typical braindead person on the street. They actually use a decent but not crazily advanced vocabulary."
I mean, "tapestry" is a great word for something that is interconnected. Why not use it?
Many of these are standard fare in legal writing.
Negative parallelism is a staple of briefs. "This case is not about free speech. It is about fraud." It does real work when you're contesting the other side's framing.
Tricolons and anaphora are used as persuasion techniques for closing arguments and appellate briefs.
Short punchy fragments help in persuasive briefs where judges are skimming. "The statute is unambiguous."
As with the em dash - let's not throw the baby out with the bath water.
> (let's play cat and mouse!).
No thanks, I hate this large scale social experiment
You know how no one ever wrote their own software and then generative AI came along and suddenly we could have app meals home-cooked by barefoot developers? (The use of such cottagecore terminology for a process that requires being an ongoing client of a hundred-gigabuck, planet-burning megacorporation rubs me in many wrong ways.)
If AI finally gets rid of the thing that drove me nuts for years: "leverage" as a verb mean roughly "to use"—when no human intervention seems to work, then I shall be over-the-moon happy. I once worked at a place where this particular word was lever—er, used all the damn time and I'd never encountered something so NPC-ish. I felt like I was on The Twilight Zone. I could've told you way back then that you sounded like a bot doing that, now people might actually believe me and thank god.
I will stick by the em dashes however. And I might just start using arrows too. Compose - > → right arrow. Not even difficult.
This kills the headline baiting tech blogger.
Wikipedia also has an exhaustive guide, though it's not fun finding tropes you use yourself (I'm very guilty of the false range "from X to Y" thing):
https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing
Another one that seems impossible for LLMs to avoid: breaking article into a title and a subtitle, separated by a colon. Even if you explicitly tell it not to, it'll do it.