An evaluation of Convolutional Neural Networks for music classification using spectrograms

Costa, Yandre M. G.; Oliveira, Luiz S.; Silla, Carlos N.

doi:10.1016/j.asoc.2016.12.024

Cited by 202 publications

(110 citation statements)

References 23 publications

Supporting

Mentioning

107

Contrasting

Unclassified

Order By: Relevance

“…These studies used a sound spectrogram image from the audio waveform. Our proposed method is entirely different from previous methods [20,21,22]. Our method does not lose the information included in the raw audio waveform.…”

Section: Bit Pattern Imagementioning

confidence: 97%

Audio Classification of Bit-Representation Waveform

Okawa

Saito²,

Sawada

et al. 2019

Interspeech 2019

View full text Add to dashboard Cite

This study investigated the waveform representation for audio signal classification. Recently, many studies on audio waveform classification such as acoustic event detection and music genre classification have been published. Most studies on audio waveform classification have proposed the use of a deep learning (neural network) framework. Generally, a frequency analysis method such as Fourier transform is applied to extract the frequency or spectral information from the input audio waveform before inputting the raw audio waveform into the neural network. In contrast to these previous studies, in this paper, we propose a novel waveform representation method, in which audio waveforms are represented as a bit sequence, for audio classification. In our experiment, we compare the proposed bit representation waveform, which is directly given to a neural network, to other representations of audio waveforms such as a raw audio waveform and a power spectrum with two classification tasks: one is an acoustic event classification task and the other is a sound/music classification task. The experimental results showed that the bit representation waveform achieved the best classification performance for both the tasks.

show abstract

Section: Bit Pattern Imagementioning

confidence: 97%

Audio Classification of Bit-Representation Waveform

Okawa

Saito²,

Sawada

et al. 2019

Interspeech 2019

View full text Add to dashboard Cite

show abstract

“…CNNs have been exploited to handle not only various tasks in the field of computer vision and multimedia [28], [29], but also the tasks of music information retrieval such as genre classification [30], acoustic event detection [31], automatic music tagging [32]. Generally speaking, when lacking computational power and large annotated datasets, it is preferred to directly use pre-trained CNNs such as VGG16 [28] to extract features [20] [31], or further combine it with fully-connected layers to extract semantic features [19][23] [31].…”

Section: A Convolutional Neural Network (Cnns)mentioning

confidence: 99%

Deep Cross-Modal Correlation Learning for Audio and Lyrics in Music Retrieval

Tang

Raposo

et al. 2019

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

Deep cross-modal learning has successfully demonstrated excellent performances in cross-modal multimedia retrieval, with the aim of learning joint representations between different data modalities. Unfortunately, little research focuses on cross-modal correlation learning where temporal structures of different data modalities such as audio and lyrics are taken into account. Stemming from the characteristic of temporal structures of music in nature, we are motivated to learn the deep sequential correlation between audio and lyrics. In this work, we propose a deep cross-modal correlation learning architecture involving two-branch deep neural networks for audio modality and text modality (lyrics). Different modality data are converted to the same canonical space where inter modal canonical correlation analysis is utilized as an objective function to calculate the similarity of temporal structures. This is the first study on understanding the correlation between language and music audio through deep architectures for learning the paired temporal correlation of audio and lyrics. Pre-trained Doc2vec model followed by fully-connected layers (fully-connected deep neural network) is used to represent lyrics. Two significant contributions are made in the audio branch, as follows: i) pretrained CNN followed by fully-connected layers is investigated for representing music audio. ii) We further suggest an end-toend architecture that simultaneously trains convolutional layers and fully-connected layers to better learn temporal structures of music audio. Particularly, our end-to-end deep architecture contains two properties: simultaneously implementing feature learning and cross-modal correlation learning, and learning joint representation by considering temporal structures. Experimental results, using audio to retrieve lyrics or using lyrics to retrieve audio, verify the effectiveness of the proposed deep correlation learning architectures in cross-modal music retrieval.

show abstract

“…Penelitian tentang penggolongan lagu sudah dilakukan oleh Yandre M.G. Costa, dkk pada jurnal yang berjudul An evaluation of Convolutional Neural Networks for music classification using spectrograms [3]…”

Section: Pendahuluanunclassified

Penggolongan Lagu Berdasarkan Spektogram Dengan Convolution Neural Network

2018

View full text Add to dashboard Cite

AbstrakSaat ini terdapat banyak lagu yang sudah diproduksi di dunia. Lagu-lagu tersebut digolongkan ke dalam genre berbeda. Ada berbagai macam genre mulai dari pop, rock, classic, reggae, dubstep, dan lain-lain. Perbedaan genre ini disebabkan adanya ketidaksamaan melodi, ketukan, intonasi, dan ekspresi pada masing-masing genre. Saat ini terdapat banyak metode yang digunakan untuk mengenali sebuah lagu, di antaranya audioprint, penggolongan genre, pengenalan ketukan lagu, pengenalan lirik lagu, dan lain-lain. Metode yang dipakai selama ini menggunakan database dengan ciri dari jutaan lagu. Salah satu metode lain adalah dengan mengembangkan sistem identifikasi lagu dengan suatu jaringan saraf terlatih. Penelitian ini akan membahas perancangan sebuah sistem untuk menggolongan lagu berdasarkan spektogram. Masukan sistem berupa lagu dengan format audio MP3 yang diubah ke dalam bentuk spektogram kemudian dilatih menggunakan Convolutional Neural Network. Ciri lagu akan diperoleh kemudian diklasifikan ke dalam lima genre berbeda yaitu pop, rock, classic, dubstep, dan reggae. Berdasarkan hasil pelatihan dan pengujian dengan filter 3x3 didapat nilai akurasi penggolongan lagu sebesar 100% pada 750 data latih dan 98% pada 50 lagu data uji. Algoritme pembelajaran terbaik pada pelatihan dengan filter yang sama adalah algoritme Adam yang lebih cepat dibandingkan dengan Adadelta, Adagrad, dan SGD. AbstractCurrently there are many songs that have been produced in the world. The songs are classified into different genres. There are various genres ranging from pop, rock, classical, reggae, dubstep, and others. This genre difference occurs in melodic inequality, tapping, intonation, and expression in each genre. Currently there are many methods used to classify a song, such as audioprint, genre classification, song recognition, song lyrics, etc. The method used so far uses a database with features from millions of songs. The other best method is to create a song classification system with a neural network system. This research will discuss the design of a system for classifying songs based on spectrograph. By using spectrograph of songs to be used for training using Convolutional Neural Network. Characteristics of songs will be classified into five genres of pop, rock, classic, dubstep, and reggae. Based on the results of training and testing with 3x3 filter obtained classification for 750 training data by 100% and 50 songs of test data by 98%. The best learning algorithm in training with the same filter is Adam algorithm which is faster compared to Adadelta, Adagrad, and SGD.

show abstract

An evaluation of Convolutional Neural Networks for music classification using spectrograms

Cited by 202 publications

References 23 publications

Audio Classification of Bit-Representation Waveform

Audio Classification of Bit-Representation Waveform

Deep Cross-Modal Correlation Learning for Audio and Lyrics in Music Retrieval

Penggolongan Lagu Berdasarkan Spektogram Dengan Convolution Neural Network

Contact Info

Product

Resources

About