Modeling Timbre Similarity of Short Music Clips

Abstract

There is evidence from a number of recent studies that most listeners are able to extract information related to song identity, emotion, or genre from music excerpts with durations in the range of tenths of seconds. Because of these very short durations, timbre as a multifaceted auditory attribute appears as a plausible candidate for the type of features that listeners make use of when processing short music excerpts. However, the importance of timbre in listening tasks that involve short excerpts has not yet been demonstrated empirically. Hence, the goal of this study was to develop a method that allows to explore to what degree similarity judgments of short music clips can be modeled with low-level acoustic features related to timbre. We utilized the similarity data from two large samples of participants: Sample I was obtained via an online survey, used 16 clips of 400 ms length, and contained responses of 137,339 participants. Sample II was collected in a lab environment, used 16 clips of 800 ms length, and contained responses from 648 participants. Our model used two sets of audio features which included commonly used timbre descriptors and the well-known Mel-frequency cepstral coefficients as well as their temporal derivates. In order to predict pairwise similarities, the resulting distances between clips in terms of their audio features were used as predictor variables with partial least-squares regression. We found that a sparse selection of three to seven features from both descriptor sets—mainly encoding the coarse shape of the spectrum as well as spectrotemporal variability—best predicted similarities across the two sets of sounds. Notably, the inclusion of non-acoustic predictors of musical genre and record release date allowed much better generalization performance and explained up to 50% of shared variance (R2) between observations and model predictions. Overall, the results of this study empirically demonstrate that both acoustic features related to timbre as well as higher level categorical features such as musical genre play a major role in the perception of short music clips.

A snippet in a snippet: Development of the Matryoshka principle for the construction of very short musical stimuli (plinks)

“…Müller (2015) explained this by pointing to the potential of MFCCs to capture aspects like instrumentation and timbre. In line with this view, Müllensiefen and Siedenburg (2017) used MFCCs as external criteria for the validation of (dis)similarity ratings in music psychology.…”

Interval and Ratio Scaling of Spectral Audio Descriptors

“…Audio features have been widely used in timbre research for explaining quantitatively the dimensions of timbre spaces (Grey and Gordon, 1978 ; Iverson and Krumhansl, 1993 ; McAdams et al, 1995 ; Lakatos, 2000 ), affective ratings (Laurier et al, 2009 ; Farbood and Price, 2017 ; McAdams et al, 2017 ), and the perceptual similarity of short music clips (Siedenburg and Müllensiefen, 2017 ). Most often, the spectral features are derived from statistical computations on a spectrogram, whereas the temporal features are usually extracted from the raw waveform.…”

The Roles of Absolute Pitch and Timbre in Plink Perception

“…However, Schellenberg et al (1999) argued, based on the facts that their plinks were too brief to present tempo or relative pitch information and that listeners recognized reversed plinks less accurately than regular plinks (despite them sharing acoustic features related to absolute pitch), that listeners must have utilized spectral timbral cues to identify the plinks. Siedenburg and Mu ¨llensiefen (2017) developed statistical models to determine which acoustic factors enable listeners to categorize unfamiliar plinks. The authors proposed that multiple timbre-related acoustic cues contribute to listeners' abilities to judge plink similarity.…”