A Query-by-Singing System for Retrieving Karaoke Music

Abstract

This paper investigates the problem of retrieving karaoke music using query-by-singing techniques. Unlike regular CD music, where the stereo sound involves two audio channels that usually sound the same, karaoke music encompasses two distinct channels in each track: one is a mixture of the lead vocals and background accompaniment, and the other consists of accompaniment only. Although the two audio channels are distinct, the accompaniments in the two channels often resemble each other. We exploit this characteristic to: i) infer the background accompaniment for the lead vocals from the accompaniment-only channel, so that the main melody underlying the lead vocals can be extracted more effectively; and ii) detect phrase onsets based on the Bayesian information criterion (BIC) to predict the onset points of a song where a user's sung query may begin, so that the similarity between the melodies of the query and the song can be examined more efficiently. To further refine extraction of the main melody, we propose correcting potential errors in the estimated sung notes by exploiting a composition characteristic of popular songs whereby the sung notes within a verse or chorus section usually vary no more than two octaves. In addition, to facilitate an efficient and accurate search of a large music database, we employ multiple-pass dynamic time warping (DTW) combined with multiple-level data abstraction (MLDA) to compare the similarities of melodies. The results of experiments conducted on a karaoke database comprised of 1071 popular songs demonstrate the feasibility of query-by-singing retrieval for karaoke music.Index Terms-Bayesian information criterion, dynamic time warping, karaoke, music information retrieval, query-by-singing.

Follow That Tune-Adaptive Approach to DTW-based Query-by-Humming System

“…The query may be transposed by, e.g., all possible numbers of semitones within the octave (Yu et al, 2008) or from −5 to +5 semitones in half-of-the-semitone steps (Jang et al, 2011). Various numbers of repetitions may be considered but in any way this is clearly a brute-force approach which increases the computational complexity significantly.…”

“…In general, the proposed tune follower and its adaptive variant enable to efficiently refine the results without computationally complex methods such as repeating the DTW for all possible transpositions (Yu et al, 2008). It should be noted that they can be used independently from efficient indexing techniques (Zhu, Shasha, 2003; Keogh, 2002) or note-based approximate algorithms to increase the speed and reliability of a QBSH-based search engine.…”

A Survey on Query-by-Example based Music Information Retrieval

“…Common input query types in MIR systems are example [7][8][9][10], singing [11,12] and humming [11,13]. The popularity of multimedia devices (e.g.…”

Estimating timing and channel distortion across related signals

“…The comprehensive online music discography Discogs.com lists over 200,000 releases containing an instrumental mix but only about 40,000 which include an a cappella mix. The availability of these separated mixes are crucial in the creation and performance of some genres of music [3,4,5]. These instrumental and a cappella versions can also be used as ground-truth for vocal removal or isolation algorithms [6].…”

“…A simple approach is proposed in [5] where an optimally shifted and scaled instrumental mix is subtracted from the complete mix in the time or frequency domain in attempt to obtain a (previously unavailable) a cappella mix. However, this approach does not cover the more general case where different mixes may be extracted from different media (e.g.…”