End-to-end Music Remastering System Using Self-supervised and Adversarial Training

Abstract

Mastering is an essential step in music production, but it is also a challenging task that has to go through the hands of experienced audio engineers, where they adjust tone, space, and volume of a song. Remastering follows the same technical process, in which the context lies in mastering a song for the times. As these tasks have high entry barriers, we aim to lower the barriers by proposing an endto-end music remastering system that transforms the mastering style of input audio to that of the target. The system is trained in a selfsupervised manner, in which released pop songs were used for training. We also anticipated the model to generate realistic audio reflecting the reference's mastering style by applying a pre-trained encoder and a projection discriminator. We validate our results with quantitative metrics and a subjective listening test and show that the model generated samples of mastering style similar to the target.

Exploiting Negative Preference in Content-based Music Recommendation with Contrastive Learning

“…Furthermore, musical feature extractors utilizing advanced deep learning techniques have also been proposed. [36] and [21] use the idea of contrastive learning, and [11] use the idea of multi-scale VQ-VAE to extract low-level features of music. Accordingly, content-based music recommendations adopting these features are being suggested, demonstrating their practical applicability [10].…”

“…Considering that our work attempts to relate the contents with user feedback, we use low-level features following [37]. As representation learning has been actively studied in recent years, [11,21,36] proposed models trained in a self-supervised manner that produce novel music representations. There are also front-end models used in automatic music tagging [8,31], but they are models trained to classify music into a limited, discrete range of descriptions.…”

“…We use different feature extractors to evaluate the stability of our proposed method despite its varying performance according to each music representation. Our work uses the framework of CLMR [36], MEE [21], and Jukebox [11] as front-end musical feature extractors. CLMR is a method of extracting musical features based on the idea of SimCLR [5], which performs contrastive learning by designating different sections of the same song as positive samples and sections of different songs as negative samples.…”

“…For the training, the features of each song are represented using existing works of musical feature extraction. To guarantee the stability of our work, we used three different musical feature extraction methods -Contrastive Learning of Musical Representations (CLMR) [36], Music Effects Encoder (MEE) [21], and Jukebox [11]. Our models will finally be evaluated on the test set with accuracy, precision, recall, area under the receiver operating characteristic curve (AUROC), and false positive rate.…”