The crazy part is that none of this is unexpected.

This was exactly the reason why GPT-2 was restricted for general release in 2019.

Check out section 4 - https://cdn.openai.com/GPT_2_August_Report.pdf