A Tutorial on Deep Learning for Music Information Retrieval

Abstract

Following their success in Computer Vision and other areas, deep learning techniques have recently become widely adopted in Music Information Retrieval (MIR) research. However, the majority of works aim to adopt and assess methods that have been shown to be effective in other domains, while there is still a great need for more original research focusing on music primarily and utilising musical knowledge and insight. The goal of this paper is to boost the interest of beginners by providing a comprehensive tutorial and reducing the barriers to entry into deep learning for MIR. We lay out the basic principles and review prominent works in this hard to navigate field. We then outline the network structures that have been successful in MIR problems and facilitate the selection of building blocks for the problems at hand. Finally, guidelines for new tasks and some advanced topics in deep learning are discussed to stimulate new research in this fascinating field.

A Comparison of Audio Signal Preprocessing Methods for Deep Neural Networks on Music Tagging

“…N refers to the number of feature maps (which is set to 32 for all layers in this paper) while W refers to the weights matrix of the fully-connected output layer.) (3,3)) and (4, 4) Fully-connected layer (50) output (50) [7] and Kapre [8]. Total 224,242 tracks are used and split into train/validation/test sets, 201,672/12,633/28,537 tracks respectively.…”

“…As a result, decibel-scaled melspectrograms always outperform the linear versions as shown in Fig 5, where the same-coloured bars should be compared across within {1 vs. 2} and {1s vs. 2s}. Colours indicate normalization schemes while {1 vs. 1s} and {2 vs. 2s} compare effect, both of which are explained in Section 2.3 3 . Compared to the performance differences while controlling the training set size (the pink bar charts on the right of Figure 5) the additional work introduced by not using decibel scaling can be roughly estimated by comparing these scores to those networks when the training data size is limited (in pink).…”

“…STFT and melspectrogram have been the most popular input representations for music classification [3]. Although sample-based deep learning methods have been introduced, 2-dimensional represenations would be still useful in the near future for efficient training.…”

“…For example, mel-spectrograms have been preferred over short-time Fourier transform in many tasks [3] because it was considered to have enough information despite of its smaller size, i.e., efficient yet effective. When a time-frequency representation X is given, one of the most common preprocessing approaches is to apply logarithmic compression, i.e., log(X + α) where α can be arbitrary constants such as very small number (e.g.…”

Raw Multi-Channel Audio Source Separation using Multi- Resolution Convolutional Auto-Encoders

“…Computing the STFT to obtain features with high resolution in frequency leads to features with low resolution in time, and vise versa [6]. Most audio processing approaches prefer an auditory motivated frequency scale such as Mel, Bark, or Log scaling rather than a linear frequency scale [7], [8]. However, it is usually not easy to reconstruct the time-domain signals from those type of features.…”