Spectrogram Classification Using Dissimilarity Space

Abstract

In this work, we combine a Siamese neural network and different clustering techniques to generate a dissimilarity space that is then used to train an SVM for automated animal audio classification. The animal audio datasets used are (i) birds and (ii) cat sounds, which are freely available. We exploit different clustering methods to reduce the spectrograms in the dataset to a number of centroids that are used to generate the dissimilarity space through the Siamese network. Once computed, we use the dissimilarity space to generate a vector space representation of each pattern, which is then fed into an support vector machine (SVM) to classify a spectrogram by its dissimilarity vector. Our study shows that the proposed approach based on dissimilarity space performs well on both classification problems without ad-hoc optimization of the clustering methods. Moreover, results show that the fusion of CNN-based approaches applied to the animal audio classification problem works better than the stand-alone CNNs.

Animal Sound Classification Using Dissimilarity Spaces

“…SNNs have been applied to sound classification in [37], [38], and [39] and have the advantage over the canonical CNN in their ability to generalize. In [40], a system was developed based on dissimilarity spaces, such as that proposed in [41] for brain image classification, where a distance model was learned by training a SNN [42] on dissimilarity values. This system combined different clustering approaches to generate a dissimilarity space that was then used to train an SVM.…”

“…The system proposed in this work is similar to [40] in that it generates a dissimilarity space from the training set using an SNN to define a distance function from the input spectrograms. The objective at this point in the process is to maximize the distance separating the patterns of the different classes.…”

“…The objective at this point in the process is to maximize the distance separating the patterns of the different classes. Unlike [40], however, four different CNN architectures are selected for the twin classifiers, and both the original input spectrograms and the spectrograms processed by Heterogeneous Auto-Similarities of Characteristics [45] make up the inputs to the SNNs. In the testing phase, the different SNNs compare two samples to obtain a measure of their dissimilarity.…”

“…The approach taken in this paper is evaluated on the same animal vocalization datasets as in [40], i.e., on cats [34] and birds [7]. In addition, the system is tested on the challenging benchmark dataset for Environmental Sound Classification (ESC-50) [46].…”

“…In addition, the system is tested on the challenging benchmark dataset for Environmental Sound Classification (ESC-50) [46]. As in [40], an ensemble of SVMs trained on different dissimilarity spaces generated by changing the value of k in the clustering approaches and network topologies are combined by sum rule. Performance is compared with both the state-of-the-art as well as with fusions with the state-of-the-art.…”