Self-Consuming Generative Models go MAD

Abstract

Seismic advances in generative AI algorithms have led to the temptation to use AI-synthesized data to train next-generation models. Repeating this process creates autophagous (“self-consuming”) loops whose properties are poorly understood. We conduct a thorough analysis using state-of-the-art generative image models of three autophagous loop families that differ in how they incorporate fixed or fresh real training data and whether previous generations' samples have been biased to trade off data quality versus diversity. Our primary conclusion across all scenarios is that without enough fresh real data in each generation of an autophagous loop, future generative models are doomed to have their quality (precision) or diversity (recall) progressively decrease. We term this condition Model Autophagy Disorder (MAD) and show that appreciable MADness arises in just a few generations.