> The AIs will adapt to this

I don't think this is true, at least not right now, and in a way I'm actually thankful for it.

The frantic rush to chase the only potentially profitable use case for LLMs found so far (writing code) and the resulting focus on coding RLHF means models are actively becoming worse at sounding like humans.

This is my favorite example, and it's already relatively outdated: https://progress.openai.com/?prompt=10