Pretraining is full of bad writing and it doesn't really cause issues. Writing style comes from post-training. In this case it's gotten worse because they prioritized agentic abilities.

This is my personal theory for the cause of this style: Ouroboros. The official OpenAI explanation for how ChatGPT got obsessed with goblins blames it on exactly that:

---

That creates a feedback loop:

- Playful style is rewarded

- Some rewarded examples contain a distinctive lexical tic.

- The tic appears more often in rollouts.

- Model-generated rollouts are used for supervised fine-tuning (SFT).

- The model gets even more comfortable producing the tic.

I'm not super sure if this is true (yet?). I think that these newer LLMs are trained on results (the agent got some code to run with minimal prompting), and not on text. (I think this is called RLVR.)