A Comparison of Deep Learning Methods for Timbre Analysis in Polyphonic Automatic Music Transcription

Abstract

Automatic music transcription (AMT) is a critical problem in the field of music information retrieval (MIR). When AMT is faced with deep neural networks, the variety of timbres of different instruments can be an issue that has not been studied in depth yet. The goal of this work is to address AMT transcription by analyzing how timbre affect monophonic transcription in a first approach based on the CREPE neural network and then to improve the results by performing polyphonic music transcription with different timbres with a second approach based on the Deep Salience model that performs polyphonic transcription based on the Constant-Q Transform. The results of the first method show that the timbre and envelope of the onsets have a high impact on the AMT results and the second method shows that the developed model is less dependent on the strength of the onsets than other state-of-the-art models that deal with AMT on piano sounds such as Google Magenta Onset and Frames (OaF). Our polyphonic transcription model for non-piano instruments outperforms the state-of-the-art model, such as for bass instruments, which has an F-score of 0.9516 versus 0.7102. In our latest experiment we also show how adding an onset detector to our model can outperform the results given in this work.

Musicaiz: A python library for symbolic music generation, analysis and visualization

“…Eval. We can evaluate music generation from a subjective perspective [22] or an objective method. This submodule corresponds to the implementation of Yang and Lerch's evaluation method [23] for the objective evaluation of music generation.…”

Recognition of musical beat and style and applications in interactive humanoid robot

“…The difference between RNN and the traditional NN is that RNN has the concept of timing, and the state of the next moment will be affected by the current state. Some researchers also call recurrent networks deep networks, whose depth can be shown in input, output, and time-depth (Hernandez-Olivan et al, 2021 ; Parmiggiani et al, 2021 ). The RNN structure is given in Figure 2 .…”

Acoustic Descriptors for Characterization of Musical Timbre Using the Fast Fourier Transform

“…This is important for problems such as Automatic Music Transcription (AMT) [8]. It has been identified that, for audio recordings, it is necessary to have efficient systems that identify the different musical timbres with high precision and quantitatively [9].…”

“…that can be computationally extracted from the statistical analysis in the digitization of the spectrum (FFT). This focuses mainly on the statistical and mathematical characterization of the maximums in the FFT, such as the mean value in frequency (centroid) and amplitude, standard deviation, kurtosis, roots or poles of the distribution, arithmetic and geometric sequences in frequency, mean values and mean-squares of the amplitudes, among others [1,9,[16][17][18][19][20][21]. Currently, there is no consensus on which and how many acoustic descriptors are to characterize musical timbre.…”