MS-SincResNet: Joint Learning of 1D and 2D Kernels Using Multi-scale SincNet and ResNet for Music Genre Classification

Abstract

In this study, we proposed a new end-to-end convolutional neural network, called MS-SincResNet, for music genre classification. MS-SincResNet appends 1D multi-scale SincNet (MS-SincNet) to 2D ResNet as the first convolutional layer in an attempt to jointly learn 1D kernels and 2D kernels during the training stage. First, an input music signal is divided into a number of fixed-duration (3 seconds in this study) music clips, and the raw waveform of each music clip is fed into 1D MS-SincNet filter learning module to obtain three-channel 2D representations. The learned representations carry rich timbral, harmonic, and percussive characteristics comparing with spectrograms, harmonic spectrograms, percussive spectrograms and Mel-spectrograms. ResNet is then used to extract discriminative embeddings from these 2D representations. The spatial pyramid pooling (SPP) module is further used to enhance the feature discriminability, in terms of both time and frequency aspects, to obtain the classification label of each music clip. Finally, the voting strategy is applied to summarize the classification results from all 3-second music clips. In our experimental results, we demonstrate that the proposed MS-SincResNet outperforms the baseline SincNet and many well-known hand-crafted features. Considering individual 2D representation, MS-SincResNet also yields competitive results with the state-of-the-art methods on the GTZAN dataset and the ISMIR2004 dataset. The code is

Broadcast Swin Transformer for Music Genre Classification

“…It shows that BST achieves much better classification performance on GTZAN than its predecessors. [6] 93.9 MS-SincResNet [8] 91.5 S3T [9] 81.1 AST with IPET [10] 90.8 PIPMN [11] 93.2 BST 99.0…”

“…Secondly, the number of audios in GTZAN is small, while the length of each audio is long and the feature information is quite rich, so it's easy to result in overfitting. In addition, the MS-SincNet structure proposed by Chang et al can extract features with rich timbres, harmonics and percussions from audios [8], which is more effective than hand-crafted features such as spectrogram, Mel spectrogram, percussion spectrogram and harmonic spectrogram. Thus, this paper's preprocessing method for extracting audio features by Mel-spectrogram needs to be improved.…”

“…Gong et al introduced Transformer into audio classification for the first time and proposed Audio Spectrogram Transformer (AST) [7], which purely depends on attention and provides a new solution for MGC. Chang et al proposed a novel end-to-end model called MS-SincResNet, attempting to cooperatively learn 1D and 2D kernels during the training phase [8]. The model achieved 91.49% accuracy on GTZAN.…”

A Machine Anomalous Sound Detection Method Using the lMS Spectrogram and ES-MobileNetV3 Network

“…The SincNet filters used in this architecture are the inverse Fourier transform of some rectangular band-pass filters, inspired by band-pass filters in the field of signal processing. Chang et al [30] extended this concept by designing a multi-scale SincNet (MS-SincNet) 2D representation extraction network based on the SincNet filter. The MS-SincNet network autonomously learns filter parameters and exhibits a degree of anti-noise ability.…”