Regeneration Learning: A Learning Paradigm for Data Generation

Abstract

Machine learning methods for conditional data generation usually build a mapping from source conditional data X to target data Y . The target Y (e.g., text, speech, music, image, video) is usually high-dimensional and complex, and contains information that does not exist in source data, which hinders effective and efficient learning on the source-target mapping. In this paper, we present a learning paradigm called regeneration learning for data generation, which first generates Y (an abstraction/representation of Y ) from X and then generates Y from Y . During training, Y is obtained from Y through either handcrafted rules or selfsupervised learning and is used to learn X → Y and Y → Y . Regeneration learning extends the concept of representation learning to data generation tasks, and can be regarded as a counterpart of traditional representation learning, since 1) regeneration learning handles the abstraction (Y ) of the target data Y for data generation while traditional representation learning handles the abstraction (X ) of source data X for data understanding; 2) both the processes of Y → Y in regeneration learning and X → X in representation learning can be learned in a self-supervised way (e.g., pre-training); 3) both the mappings from X to Y in regeneration learning and from X to Y in representation learning are simpler than the direct mapping from X to Y . We show that regeneration learning can be a widely-used paradigm for data generation (e.g., text generation, speech recognition, speech synthesis, music composition, image generation, and video generation) and can provide valuable insights into developing data generation methods.