Tonal representations for music retrieval: from version identification to query-by-humming

Abstract

In this study we compare the use of different music representations for retrieving alternative performances of the same musical piece, a task commonly referred to as version identification. Given the audio signal of a song, we compute descriptors representing its melody, bass line and harmonic progression using state-of-the-art algorithms. These descriptors are then employed to retrieve different versions of the same musical piece using a dynamic programming algorithm based on nonlinear time series analysis. First, we evaluate the accuracy obtained using individual descriptors, and then we examine whether performance can be improved by combining these music representations (i.e. descriptor fusion). Our results show that whilst harmony is the most reliable music representation for version identification, the melody and bass line representations also carry useful information for this task. Furthermore, we show that by combining these tonal representations we can increase version detection accuracy. Finally, we demonstrate how the proposed version identification method can be adapted for the task of query-by-humming. We propose a melody-based retrieval approach, and demonstrate how melody representations extracted from recordings of a cappella singing can be successfully used to retrieve the original song from a collection of polyphonic audio. The current limitations of the proposed approach are discussed in the context of version identification and query-by-humming, and possible solutions and future research directions are proposed.

Bridging the Gap: Enriching YouTube Videos with Jazz Music Annotations

“…In other scenarios, one may have to deal with imperfect or less specific queries. For instance, a query might be a person humming the solo, which then requires an extra step for extracting the fundamental frequency from the hummed melody (queryby-humming), see, e.g., Pauws (2002), Ryynänen and Klapuri (2008), and Salamon et al (2013). Furthermore, the query may have a different tuning, may be transposed to another key, or played with rhythmic variations.…”

“…A common preprocessing step, which targets the "polyphony gap" between query and database document, is to enhance the predominant melody in audio recordings. In Salamon et al (2013), the authors used a so-called salience representation in a query-by-humming system which led to a substantial increase in performance (Salamon and Gómez, 2012). In Balke et al (2017b), a data-driven approach is used to estimate a salience representation for jazz music recordings which showed a similar performance as the aforementioned, salience-based method.…”

Robust Singing Transcription System Using Local Homogeneity in the Harmonic Structure

“…Evaluations have been conducted using a publicly available dataset [22], [23], released for the purpose of evaluation on singing transcription. The dataset consists of 38 audio recordings of monophonic singing, recorded with a sample rate of 44.1 kHz and a 16-bit resolution.…”

Music Information Retrieval: Recent Developments and Applications

“…One of the first proposed systems is MUSART by Birmingham et al [43]. Music collections for this task were traditionally built with music scores, user hummed or tapped queries -more recently with audio signals as in the system by Salamon et al [218]. Commercial systems are also exploiting the idea of retrieving music by singing, humming or typing.…”