alt Hacker News> In other words, it's the most average, the most flat. [..] And when you think about it, the AI's training data comes from the broadest consumer distribution. It spits out that kind of data because it learned that most people preferred it.
That's not what it is! The heavy reinforcement learning that the models go through makes their writing the farthest thing possible from the "most average, flat" distribution. The distribution actually becomes very sharp due to this, unlike any writing it has seen in 99.99% of its training data! (The heavier the RL is, the more same-y the output becomes.) This is why every AI-written piece feels the same, and why people can learn to easily tell that something was written by an AI. And frontier labs make this problem even worse by how they sample[1] their logits.
[1]: https://gist.github.com/Hellisotherpeople/71ba712f9f899adcb0...
> But when I read a technical article, the most important thing, unlike a novel, isn't the prose style.
Exactly! Which is why I think there's no need to "enhance" it with AI.
Thank you for pointing out something I didn't know. That said, I do think there are some issues with it, just as you mentioned.
I think it's a simulacrum and simulation problem. I recently had a chance to teach some students, and I noticed their writing was starting to resemble AI generated text. The reason was that the AI was guiding their learning, and the students ended up picking up the AI's way of speaking.
Honestly, you can't rule out that kind of issue, and the homogenization of voice is definitely a valid concern too.
Opinions on what counts as "improvement" will vary from person to person, but I do think the problems you've raised can't be completely dismissed.
Thanks for the insight, and I'll take the time to read through the links you shared. Have a good day.