I guess more important than LLMs training LLMs (which is mostly the same code over and over) is LLMs cleaning and curating training data.