logoalt Hacker News

317070 • today at 2:20 PM • 1 reply • view on HN

Language models don't have a characteristic style you can spot from a mile away. They haven't had it since before ChatGPT.

What is true, is that chat models (and by extension agents) have a very characteristic style after their LLM went through post-training. But the pre-trained language model doesn't have that style.

So if you want to write well with an LLM, you could and it is a well understood problem. It is more a matter of that not being of much interest, as it competes with having the model be good for chat interfaces.

How to have a well-aligned, RL'ed, safe or thinking model, which also writes well in long-form, is a bit an open problem. Though presumably agents could do it when they have access to a pretrained model.


Replies

SoftTalker • today at 4:13 PM

Why is writing in the style of a corporate marketing department a requirement of being good for chat interfaces? Why can't they tune them to sound more like a normal human?