Improving Singing Voice Separation Using Curriculum Learning on Recurrent Neural Networks

Abstract

Single-channel singing voice separation has been considered a difficult task, as it requires predicting two different audio sources independently from mixed vocal and instrument sounds recorded by a single microphone. We propose a new singing voice separation approach based on the curriculum learning framework, in which learning is started with only easy examples and then task difficulty is gradually increased. In this study, we regard the data providing obviously dominant characteristics of a single source as an easy case and the other data as a difficult case. To quantify the dominance property between two sources, we define a dominance factor that determines a difficulty level according to relative intensity between vocal sound and instrument sound. If a given data is determined to provide obviously dominant characteristics of a single source according to the factor, it is regarded as an easy case; otherwise, it belongs to a difficult case. Early stages in the learning focus on easy cases, thus allowing rapidly learning overall characteristics of each source. On the other hand, later stages handle difficult cases, allowing more careful and sophisticated learning. In experiments conducted on three song datasets, the proposed approach demonstrated superior performance compared to the conventional approaches.

Music Deep Learning: Deep Learning Methods for Music Signal Processing—A Review of the State-of-the-Art

“…For singing voice separation, in [47], a curriculum learning approach was considered, where the training begins with easy examples and the difficulty is steadily increased. Three different databases were tested: MIR-1K [48], ccMixter [49], and MUSDB18 [50], with the model yielding improved performance with respect to the global normalized source distortion ratio measure.…”

Deep Learning Approaches in Topics of Singing Information Processing

“…Kang et al [175] proposed a singing voice separation approach based on the curriculum learning framework, in which learning is started with only easy examples and then the task difficulty is gradually increased. They define easy examples as the ones in which one source is obviously dominant over the other, where the dominance factor depends on the relative intensity of vocals and instruments.…”

Sound Source Separation Mechanisms of Different Deep Networks Explained from the Perspective of Auditory Perception

“…The second approach develops supervised models based on the task optimization on a specific training dataset. Thanks to the development of deep learning, recently proposed deep source separation systems have made significant progress [10,[14][15][16], which has begun to perform at the human level for natural source separation. The second approach has attracted significant attention because of its excellent performance, and it is also used here to solve the problem of two source separation from monaural recordings.…”