A Relevance Score Estimation for Spoken Term Detection Based on RNN-Generated Pronunciation Embeddings

Švec, Ján; Psutka, Josef; Šmídl, Luboš; Trmal, Jan

doi:10.21437/interspeech.2017-1087

Cited by 7 publications

(13 citation statements)

References 14 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since we are targeting the large spoken archives for which the LVCSR system exists and is used for searching the IV terms, we focused on the methods where the STD is performed using the phoneme recognizer (in a structure similar to LVCSR) to search the OOV terms. The idea is not new and we used it in a mostly heuristic search presented in [2] and subsequently we adopted the approach of Siamese networks [17] to robustly estimate the term relevance scores. In this work we modified the Siamese architecture presented in [17] with the goal to simplify the network architecture and further improve the STD performance: STD process -while the original Siamese architecture was proposed to relevance score estimation only and the localization of terms was performed using the index of phoneme triplets, the proposed approach allows both to localize and score the putative hits of the searched term.…”

Section: Introductionmentioning

confidence: 99%

Spoken Term Detection and Relevance Score Estimation Using Dot-Product of Pronunciation Embeddings

Švec¹,

Šmídl²,

Psutka³

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

The paper describes a novel approach to Spoken Term Detection (STD) in large spoken archives using deep LSTM networks. The work is based on the previous approach of using Siamese neural networks for STD and naturally extends it to directly localize a spoken term and estimate its relevance score. The phoneme confusion network generated by a phoneme recognizer is processed by the deep LSTM network which projects each segment of the confusion network into an embedding space. The searched term is projected into the same embedding space using another deep LSTM network. The relevance score is then computed using a simple dot-product in the embedding space and calibrated using a sigmoid function to predict the probability of occurrence. The location of the searched term is then estimated from the sequence of output probabilities. The deep LSTM networks are trained in a self-supervised manner from paired recognition hypotheses on word and phoneme levels. The method is experimentally evaluated on MALACH data in English and Czech languages.

show abstract

Section: Introductionmentioning

confidence: 99%

Spoken Term Detection and Relevance Score Estimation Using Dot-Product of Pronunciation Embeddings

Švec¹,

Šmídl²,

Psutka³

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

show abstract

“…Subword-based ASR, on the other hand, can detect terms even though they are not in the vocabulary of the recognizer (i.e., out-of-vocabulary (OOV) terms). A more robust system could be obtained by combining both approaches [17,24,25,38,39,42,50,[67][68][69][70][71][72][73].…”

Section: Spoken Term Detectionmentioning

confidence: 99%

“…In this direction, several end-to-end ASR-free approaches for STD were proposed [13,[34][35][36]. In addition to exploring neural end-to-end approaches, deep learning is extensively used to extract representations (embeddings) of audio documents and query terms that facilitate the search [20,21,23,25].…”

Section: Spoken Term Detectionmentioning

confidence: 99%

“…The huge amount of information stored in audio and audiovisual repositories makes it necessary to develop efficient methods for search on speech (SoS). Significant research has been carried out for years in this area, and, in particular, in the tasks of spoken document retrieval (SDR) [1][2][3][4][5][6], keyword spotting (KWS) [7][8][9][10][11][12][13], spoken term detection (STD) [14][15][16][17][18][19][20][21][22][23][24][25] and query-by-example spoken term detection (QbE STD) [26][27][28][29][30][31].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Multi-Domain International Search on Speech 2020 ALBAYZIN Evaluation: Overview, Systems, Results, Discussion and Post-Evaluation Analyses

et al. 2021

View full text Add to dashboard Cite

The large amount of information stored in audio and video repositories makes search on speech (SoS) a challenging area that is continuously receiving much interest. Within SoS, spoken term detection (STD) aims to retrieve speech data given a text-based representation of a search query (which can include one or more words). On the other hand, query-by-example spoken term detection (QbE STD) aims to retrieve speech data given an acoustic representation of a search query. This is the first paper that presents an internationally open multi-domain evaluation for SoS in Spanish that includes both STD and QbE STD tasks. The evaluation was carefully designed so that several post-evaluation analyses of the main results could be carried out. The evaluation tasks aim to retrieve the speech files that contain the queries, providing their start and end times and a score that reflects how likely the detection within the given time intervals and speech file is. Three different speech databases in Spanish that comprise different domains were employed in the evaluation: the MAVIR database, which comprises a set of talks from workshops; the RTVE database, which includes broadcast news programs; and the SPARL20 database, which contains Spanish parliament sessions. We present the evaluation itself, the three databases, the evaluation metric, the systems submitted to the evaluation, the evaluation results and some detailed post-evaluation analyses based on specific query properties (in-vocabulary/out-of-vocabulary queries, single-word/multi-word queries and native/foreign queries). The most novel features of the submitted systems are a data augmentation technique for the STD task and an end-to-end system for the QbE STD task. The obtained results suggest that there is clearly room for improvement in the SoS task and that performance is highly sensitive to changes in the data domain.

show abstract

“…On the other hand, the subword-based approach has the unique advantage that it can detect terms that consist of words that are not in the vocabulary of the recognizer, i.e., out-ofvocabulary (OOV) terms. The combination of these two approaches has been proposed in order to exploit the relative advantages of word and subword-based strategies [17,32,33,36,44,[63][64][65][66][67][68][69][70].…”

Section: Spoken Term Detection Overviewmentioning

confidence: 99%

ALBAYZIN 2018 spoken term detection evaluation: a multi-domain international evaluation in Spanish

Tejedor

Toledano

López-Otero

et al. 2019

J AUDIO SPEECH MUSIC PROC.

View full text Add to dashboard Cite

Search on speech (SoS) is a challenging area due to the huge amount of information stored in audio and video repositories. Spoken term detection (STD) is an SoS-related task aiming to retrieve data from a speech repository given a textual representation of a search term (which can include one or more words). This paper presents a multi-domain internationally open evaluation for STD in Spanish. The evaluation has been designed carefully so that several analyses of the main results can be carried out. The evaluation task aims at retrieving the speech files that contain the terms, providing their start and end times, and a score that reflects the confidence given to the detection. Three different Spanish speech databases that encompass different domains have been employed in the evaluation: the MAVIR database, which comprises a set of talks from workshops; the RTVE database, which includes broadcast news programs; and the COREMAH database, which contains 2-people spontaneous speech conversations about different topics. We present the evaluation itself, the three databases, the evaluation metric, the systems submitted to the evaluation, the results, and detailed post-evaluation analyses based on some term properties (within-vocabulary/out-of-vocabulary terms, single-word/multi-word terms, and native/foreign terms). Fusion results of the primary systems submitted to the evaluation are also presented. Three different research groups took part in the evaluation, and 11 different systems were submitted. The obtained results suggest that the STD task is still in progress and performance is highly sensitive to changes in the data domain.

show abstract

A Relevance Score Estimation for Spoken Term Detection Based on RNN-Generated Pronunciation Embeddings

Cited by 7 publications

References 14 publications

Spoken Term Detection and Relevance Score Estimation Using Dot-Product of Pronunciation Embeddings

Spoken Term Detection and Relevance Score Estimation Using Dot-Product of Pronunciation Embeddings

The Multi-Domain International Search on Speech 2020 ALBAYZIN Evaluation: Overview, Systems, Results, Discussion and Post-Evaluation Analyses

ALBAYZIN 2018 spoken term detection evaluation: a multi-domain international evaluation in Spanish

Contact Info

Product

Resources

About