logoalt Hacker News

Animatstoday at 7:23 PM1 replyview on HN

Well, what do you expect? LLMs are trained on blithering, mostly from web sites. So you get blithering out.

There's an important point in the article, that forcing a style onto an LLM is lossy. Although he doesn't seem to mention it, forcing a style may result in the insertion of new blithering, possibly made up as a hallucination.


Replies

mjburgesstoday at 7:56 PM

I think that was a good enough explanation for gpt3.5 -- these days, labs are extremely capable of post-training phases that eclipse that kind of training phase -- and hence of choosing whatever style or tone they wish.

eg., OpenAI has gone a long way to making reasoning token-efficient by having reasoning piovot off terse langauge -- whereas anthropic appears to be doing the opposite.