Indexing music by mood: design and integration of an automatic content-based annotator

Abstract

In the context of content analysis for indexing and retrieval, a method for creating automatic music mood annotation is presented. The method is based on results from psychological studies and framed into a supervised learning approach using musical features automatically extracted from the raw audio signal. We present here some of the most relevant audio features to solve this problem. A ground truth, used for training, is created using both social network information systems (wisdom of crowds) and individual experts (wisdom of the few). At the experimental level, we evaluate our approach on a database of 1000 songs. Tests of different classification methods, configurations and optimizations have been conducted, showing that Support Vector Machines perform best for the task at hand. Moreover, we evaluate the algorithm robustness against different audio compression schemes. This fact, often neglected, is fundamental to build a system that is usable in real conditions. In addition, the integration of a fast and scalable version of this technique with the European Project PHAROS is discussed. This real world application demonstrates the usability of this tool to annotate large-scale databases. We also report on a user evaluation in the context of the PHAROS search engine, asking people about the utility, interest and innovation of this technology in real world use cases.

Semantic structures of timbre emerging from social and acoustic descriptions of music

“…A common approach in these studies has been to extract a range of features, often low-level ones such as timbre, dynamics, articulation, Mel-frequency cepstral coefficients (MFCC) and subject them to further analysis. The parameters of the actual feature extraction are dependent on the goals of the particular study; some focus on shorter musical elements, particularly the MFCC and its derivatives [21,39,40]; while others utilize more high-level concepts, such as harmonic progression [41][42][43]. In this study, the aim was to characterize the semantic structures with a combined set of non-redundant, robust low-level acoustic and musical features suitable for this particular set of data.…”

Unifying Low-Level and High-Level Music Similarity Measures

“…In contrast, research on computing high-level semantic features from low-level audio descriptors exists. In particular, in the context of MIR classification problems, genre classification [34], mood detection [35], [36], and artist identification [24] have gathered much research attention.…”

“…Descriptor class Timbral Bark bands [35], [37] MFCCs [13], [35], [37], [38] Pitch [39], pitch centroid [40] Spectral centroid, spread, kurtosis, rolloff, decrease, skewness [35], [37], [41] High-frequency content [39], [41] Spectral complexity [35] Spectral crest, flatness, flux [37], [41] Spectral energy, energy bands, strong peak, tristimulus [41] Inharmonicity, odd to even harmonic energy ratio [37] Rhythmic BPM, onset rate [35], [39], [41] Beats loudness, beats loudness bass [40] Tonal Transposed and untransposed harmonic pitch class profiles, key strength [35], [42] Tuning frequency [42] Dissonance [35], [43] Chord change rate [35] Chords histogram, equal tempered deviations, non-tempered/tempered energy ratio, diatonic strength [40] Miscellaneous Average loudness [37] Zero-crossing rate [13], [37] 1) Euclidean distance based on principal component analysis (L 2 -PCA): As a starting point, we follow the ideas proposed by Cano et al [19], and apply an unweighted Euclidean metric on a manually selected subset of the descriptors outlined above 6 . This subset includes bark bands, pitch, spectral centroid, spread, kurtosis, rolloff, decrease, skewness, high-frequency content, spectral complexity, spectral crest, flatness, flux, spectral energy, energy bands, strong peak, tristimulus, inharmonicity, odd to even harmonic energy ratio, beats loudness, beats loudness bass, untransposed harmonic pitch class profiles, key strength, average loudness, and zerocrossing rate.…”

“…This dimensionality can vary from 10 to 20 [28], [35], [50]. To preserve robustness against different audio encodings, the first 13 MFCC coefficients are taken [51].…”

“…A schema of the approach is presented in Fig. 1 For the first step we choose standard multi-class support vector machines (SVMs) [44], which are shown to be an effective tool for different classification tasks in MIR [24], [35], [36], [55], [56]. We apply SVMs to infer different groups of musical dimensions such as (i) genre and musical culture, (ii) moods and instruments, and (iii) rhythm and tempo.…”