Music Artist Classification with Convolutional Recurrent Neural Networks

Abstract

Previous attempts at music artist classification use frame level audio features which summarize frequency content within short intervals of time. Comparatively, more recent music information retrieval tasks take advantage of temporal structure in audio spectrograms using deep convolutional and recurrent models. This paper revisits artist classification with this new framework and empirically explores the impacts of incorporating temporal structure in the feature representation. To this end, an established classification architecture, a Convolutional Recurrent Neural Network (CRNN), is applied to the artist20 music artist identification dataset under a comprehensive set of conditions. These include audio clip length, which is a novel contribution in this work, and previously identified considerations such as dataset split and feature level. Our results improve upon baseline works, verify the influence of the producer effect on classification performance and demonstrate the trade-offs between audio length and training set size. The best performing model achieves an average F1 score of 0.937 across three independent trials which is a substantial improvement over the corresponding baseline under similar conditions. Additionally, to showcase the effectiveness of the CRNN's feature extraction capabilities, we visualize audio samples at the model's bottleneck layer demonstrating that learned representations segment into clusters belonging to their respective artists.

Addressing The Confounds Of Accompaniments In Singer Identification

“…This result shows that using melody contour as additional features helps SID. Our 'CRNNM+Data aug' model achieves 0.75 song-level F1 score, which is greatly higher than that (0.67) obtained by the best existing model ('CRNN+Origin') [17] for artist20. Table 1 also shows that, for all the three models, training on Data aug outperforms those trained on Origin for the song-level result, validating the effectiveness of the data augmentation method.…”

“…Specifically, we use open-unmix [12], an open-source three-layer bidirectional deep recurrent neural network for SS. Moreover, we build upon our SID model based on the implementation of a convolutional recurrent neural network made available by Nasrullah and Zhao [17], which attains the highest song-level F1-score of 0.67 on the per-album split of the artist20 dataset [18], a standard dataset for SID. As neural networks may find their own way extracting relevant features or patterns from the input, it remains to be studied whether the use of SS can improve the performance of a deep learning based SID model.…”

“…1, but without the lower melody-related branch. We follow exactly the same design (i.e., encompassing number of filters, kernel sizes, activation functions, loss function, optimizer, learning rate, etc) of [17]. We refer to this model as 'CRNN' below.…”

“…The former splits a dataset by randomly assigning songs to the three subsets, whereas the latter makes sure that songs from the same album are either in the training, validation, or the test split. It has been known [17] that song-split may leak production details associated with an album over the training and testing subsets, giving an SID model additional clues for classification. Accordingly, the accuracy for song-split may be overly optimistic and tends to be higher than that of album-split.…”

“…Following [17], we cut the songs into 5-sec segments for training a 20-class classification model. The final prediction result for a song is made by majority voting from the persegment results.…”