Vocabulary-independent indexing of spontaneous speech

Abstract

We present a system for vocabulary-independent indexing of spontaneous speech, i.e., neither do we know the vocabulary of a speech recording nor can we predict which query terms for which a user is going to search. The technique can be applied to information retrieval, information extraction, and data mining. Our specific target is search in recorded conversations in the office/information-worker scenario-teleconferences, meetings, presentations, and voice mails.The focus of this paper is on how to index phonetic lattices. We will show that an index should provide expected term frequencies (ETFs) of query terms. Since, at indexing time, it is unknown which phoneme sequences constitute valid query terms, we will introduce an approximation of ETFs of a query's phoneme sequence by -gram phoneme language models, which are estimated on lattices and organized in an inverted index-like structure for fast access. We will discuss ranking, estimation, and integration of phoneme/word hybrid approaches. Compared with an unindexed baseline without approximation, our approximation leads only to a 3.4% relative loss of search accuracy on the Linguistic Data Consortium (LDC) voicemail task.We also propose a two-stage method for locating individual keyword occurences using the above method as a fast match. A 20-times speedup is achieved over unindexed search at under a 2-point accuracy loss.Last, we will briefly introduce a prototype applet based on the above techniques.

Query-Driven Strategy for On-the-Fly Term Spotting in Spontaneous Speech

“…A similar issue has been addressed in a different context by (Yu et al, 2005;Allauzen et al, 2004). The authors proposed to handle OOV queries in an audio search task.…”

“…For all these detection tasks, performances reported in the literature are quite good on clean conditions, especially on broadcast news data that were largely used for speech processing system benchmarking (Fiscus et al, 2007;Garofolo et al, 2000). In more difficult conditions, such as noisy or spontaneous speech, performances are dramatically degraded by recognition errors (Pinto et al, 2008;Yu et al, 2005;Saraclar, 2004).…”

“…Other authors combine complementary acoustic scoring methods, for example Gaussian mixture models (GMM) and multilayer perceptrons (MLP)-based estimators (Pinto et al, 2007;Bourlard et al, 1994). In (Yu et al, 2005), the authors propose, in the context of the STD task, to estimate the scores of the OOV by combining the posterior probabilities of their phonetic substrings. Therefore, many of the fast wordspotters are based on off-line phonetic matching.…”