Główne pojęcia
Generative models trained on their own outputs are prone to degeneration, leading to either collapse into a small subset of outputs or uniform distribution over a large set of outputs, unless a sufficient amount of external data is introduced at each iteration.
Streszczenie
The paper investigates the stability and long-term behavior of generative models that are trained in a closed-loop fashion, where the data they generate is fed back into the training process. This is a common scenario as generative models like large language models and diffusion models become widely adopted and their outputs are incorporated into shared online content.
The key insights are:
The authors define a class of "generative closed-loop learning models with temperature" that captures many real-world scenarios. The temperature parameter controls the randomness of the model's sampling.
Using tools from dynamical systems and control theory, the authors provide a theoretical analysis of the closed-loop learning dynamics. They show that modulating the sampling with temperature leads to the degeneration of the learning process, regardless of the temperature regime.
The authors characterize the type of degeneration depending on the temperature regime. In the high temperature regime, the generative distribution collapses to a small set of outputs. In the low temperature regime, the distribution becomes uniform over a large set of outputs.
As the models degenerate, so do their datasets, consequently losing any knowledge they originally contained, unless that initial dataset is preserved and re-introduced purposefully. This predicts that without preserving a copy of the pre-generative-models internet, eventually no model will be able to be trained effectively using the internet as a data source.
The results hold even when a limited amount of external data is introduced at each training iteration, as long as the proportion of synthetic data eventually dominates.
Statystyki
The paper does not contain any key metrics or important figures to support the author's key logics.
Cytaty
The paper does not contain any striking quotes supporting the author's key logics.