Keyword spotting in singing with duration-modeled HMMs

Abstract

Keyword spotting in speech is a very well-researched problem, but there are almost no approaches for singing. Most speech-based approaches cannot be applied easily to singing because the phoneme durations in singing vary a lot more than in speech, especially the vowel durations. To represent expected phoneme durations, several duration modeling techniques have been developed over the years in the field of ASR. To the best of our knowledge, these approaches have not been used for keyword spotting yet. In this paper, we present a new approach for keyword spotting in singing. We first extract various features (MFCC, TRAP, PLP, RASTA-PLP) and generate phoneme posteriograms from these features. We then perform keyword spotting on these posteriograms using keyword-filler HMMs and test two different duration modeling techniques on these HMMs: Explicit-duration modeling and Post-processor duration modeling. We evaluate our approach on a small singing data set without accompaniment

Singing Voice Phoneme Segmentation by Hierarchically Inferring Syllable and Phoneme Onset Positions

“…The second topic related to our research is the singing voice lyrics-to-audio alignment. Most of these works [11,12,13,14,15,16,17,18] used the forced alignment method accompanied by music-related techniques. Loscos et al [12] used MFCCs with additional features and also explored specific HMM topologies.…”

“…Iskandar et al [15] constrained the alignment by using musical note length distribution. Gong et al [16], Kruspe [17], Dzhambazov and Serra [18] all used sylla-ble/phoneme duration extracted from the musical score and decoded the alignment path by duration-explicit HMM models. Chien et al [19] introduced an approach based on vowel likelihood models.…”

Audio to score matching by combining phonetic and duration information

“…Because of the implicity of the Markovian state occupancy, the phonetic duration distribution introduced in section 3.3 can not be imposed. Kruspe [12] presents two duration modeling techniques for HMMs: Hidden semi-markov model (HSMM) and post-processor duration model.…”

“…The post-processor duration model was first introduced by Juang et al [10]. It was then experimentally proved in Kruspe's paper [12] that this duration model works better than HSMMs for the keyword spotting task in English pop singing voice. The post-processor duration model uses the original HMMs Viterbi algorithm -therefore, during the decoding process no explicit occupancy duration distribution is imposed.…”

“…The main goal of this task is to search for one or more keywords in a singing query. The system proposed by Kruspe [12] searches for a specific singing keyword on the resulting phoneme observations. 1 The topology of the HMM is defined by the recognition network.…”