Deep Learning Approaches in Topics of Singing Information Processing

Abstract

Singing, the vocal production of musical tones, is one of the most important elements of music. Addressing the needs of real-world applications, the study of technologies related to singing voices has become an increasingly active area of research. In this paper, we provide a comprehensive overview of the recent developments in the field of singing information processing, specifically in the topics of singing skill evaluation, singing voice synthesis, singing voice separation, and lyrics synchronization and transcription. We will especially focus on deep learning approaches including modern representation learning techniques for singing voices. We will also provide an overview of contributions in public datasets for singing voice research.

Music Deep Learning: Deep Learning Methods for Music Signal Processing—A Review of the State-of-the-Art

“…A review of DL for speech recognition is available in [6], though the focus is not on music signals. For singing information processing, [29] reviews several aspects, like singing skill evaluation, singing voice synthesis, singing voice separation, lyric synchronization, and transcription. Specifically for singing voice detection, the review in [20] investigates the traditional and deep learning techniques available.…”

PoLyScriber: Integrated Fine-Tuning of Extractor and Lyrics Transcriber for Polyphonic Music

“…The extraction-transcription approach makes use of a singing voice separation (SVS) front end to separate singing vocals from polyphonic music. There are generally two approaches for SVS, namely waveform-based and spectrogram-based approaches [23]. The waveform-based approach directly takes time-domain polyphonic music signal as input and separately output singing voice and music signals [24], [25], [26], [27].…”

“…The singing vocal carries the lyrics of a song, therefore lyrics transcription of polyphonic music can benefit a lot from the recent advances in the field of lyrics transcription of solosinging [23].…”

Unsupervised Single-Channel Singing Voice Separation with Weighted Robust Principal Component Analysis Based on Gammatone Auditory Filterbank and Vocal Activity Detection

“…Singing voice separation (SVS) has drawn a lot of interest and consideration in many downstream applications [ 1 , 2 , 3 , 4 ]. It deals with the technique of separating a singing voice or background from a mix of music, which is a crucial strategy for singer identification [ 5 , 6 ], music information retrieval [ 7 , 8 ], lyric recognition and alignment [ 9 , 10 , 11 , 12 ], song language identification [ 13 , 14 ], and chord recognition [ 15 , 16 , 17 ].…”