alt Hacker NewsAs the percentage of LLM-generated stuff in the world increases relative to natural stuff, I don't see how it's possible to continue to train new models indefinitely. At some point, won't LLMs be trained on a corpus that is majority LLM-generated data, with everything that entails both good and bad? Seems to me that singularity is just going to kick off a self-reinforcing spiral... Bigger minds than me have to have considered this, is this a legitimate concern?
Replies
How do humans deal with this?
For millennia we created documents supporting the geocentric model, all wrong! How'd we break out of that rut?
I don't know if AI are currently equipped to deal with this as humans did, but maybe they can be?
Claude reports that this is called "model collapse" and assures me Anthropic is on top of it
the models can also get better at using known data, not that it always needs more data . This was the case with math and coding.
I wonder if someone has redone the experiment of retraining LLMs on LLM output that was conducted a few years back that showed a pretty drastic degradation in quality. I imagine it's hard to perform on the closed source frontier models and the ballooning cost of training doesn't help.
In theory as the models improve their ability to ape human writing the effect should diminish, though iterative reingestion of their own output is probably still detrimental long term.