ALBAYZIN Query-by-example Spoken Term Detection 2016 evaluation

Tejedor, Javier; Toledano, Doroteo T.; López-Otero, Paula; Docío-Fernández, Laura; Proença, Jorge; Perdigão, Fernando; García-Granada, Fernando; Sanchís, Emilio; Pompili, Anna; Abad, Alberto

doi:10.1186/s13636-018-0125-9

Cited by 5 publications

(3 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ALBAYZIN evaluation campaigns comprise an internationally open set of evaluations supported by the Spanish Thematic Network on Speech Technologies (RTTH 1 ) and the ISCA Special Interest Group on Iberian Languages (SIG-IL 2 ), which have been held biennially since 2006. These evaluation campaigns provide an objective mechanism to compare different systems and are a powerful way to promote research on different speech technologies [78][79][80][81][82][83][84][85][86][87].…”

Section: Methodsmentioning

confidence: 99%

Search on speech from spoken queries: the Multi-domain International ALBAYZIN 2018 Query-by-Example Spoken Term Detection Evaluation

Tejedor

Toledano

López-Otero

et al. 2019

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

The huge amount of information stored in audio and video repositories makes search on speech (SoS) a priority area nowadays. Within SoS, Query-by-Example Spoken Term Detection (QbE STD) aims to retrieve data from a speech repository given a spoken query. Research on this area is continuously fostered with the organization of QbE STD evaluations. This paper presents a multi-domain internationally open evaluation for QbE STD in Spanish. The evaluation aims at retrieving the speech files that contain the queries, providing their start and end times, and a score that reflects the confidence given to the detection. Three different Spanish speech databases that encompass different domains have been employed in the evaluation: MAVIR database, which comprises a set of talks from workshops; RTVE database, which includes broadcast television (TV) shows; and COREMAH database, which contains 2-people spontaneous speech conversations about different topics. The evaluation has been designed carefully so that several analyses of the main results can be carried out. We present the evaluation itself, the three databases, the evaluation metrics, the systems submitted to the evaluation, the results, and the detailed post-evaluation analyses based on some query properties (within-vocabulary/out-of-vocabulary queries, single-word/multi-word queries, and native/foreign queries). Fusion results of the primary systems submitted to the evaluation are also presented. Three different teams took part in the evaluation, and ten different systems were submitted. The results suggest that the QbE STD task is still in progress, and the performance of these systems is highly sensitive to changes in the data domain. Nevertheless, QbE STD strategies are able to outperform text-based STD in unseen data domains.

show abstract

Section: Methodsmentioning

confidence: 99%

Search on speech from spoken queries: the Multi-domain International ALBAYZIN 2018 Query-by-Example Spoken Term Detection Evaluation

Tejedor

Toledano

López-Otero

et al. 2019

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Methodsmentioning

confidence: 99%

ALBAYZIN 2018 spoken term detection evaluation: a multi-domain international evaluation in Spanish

Tejedor

Toledano

López-Otero

et al. 2019

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

Search on speech (SoS) is a challenging area due to the huge amount of information stored in audio and video repositories. Spoken term detection (STD) is an SoS-related task aiming to retrieve data from a speech repository given a textual representation of a search term (which can include one or more words). This paper presents a multi-domain internationally open evaluation for STD in Spanish. The evaluation has been designed carefully so that several analyses of the main results can be carried out. The evaluation task aims at retrieving the speech files that contain the terms, providing their start and end times, and a score that reflects the confidence given to the detection. Three different Spanish speech databases that encompass different domains have been employed in the evaluation: the MAVIR database, which comprises a set of talks from workshops; the RTVE database, which includes broadcast news programs; and the COREMAH database, which contains 2-people spontaneous speech conversations about different topics. We present the evaluation itself, the three databases, the evaluation metric, the systems submitted to the evaluation, the results, and detailed post-evaluation analyses based on some term properties (within-vocabulary/out-of-vocabulary terms, single-word/multi-word terms, and native/foreign terms). Fusion results of the primary systems submitted to the evaluation are also presented. Three different research groups took part in the evaluation, and 11 different systems were submitted. The obtained results suggest that the STD task is still in progress and performance is highly sensitive to changes in the data domain.

show abstract

“…There are different speech information retrieval tasks, including spoken document retrieval (SDR), keyword spotting (KWS), spoken term detection (STD), and query-by-example spoken term detection (QbE STD). The advance of the technology in these tasks have been evaluated through different international evaluations related to SDR, STD, and QbE STD [1,2,3,4,5,6,7,8,9,10,11] for different languages (English, Arabic, Mandarin, Spanish, Japanese, and low-resource language such as Swahili, Tamil, and Vietnamese). Specifically, Spanish language has been employed with the STD/QbE STD ALBAYZIN evaluations held in 2012, 2014 and 2016 [7,8,9,10,11].…”

Section: Introductionmentioning

confidence: 99%

AUDIAS-CEU: A Language-independent approach for the Query-by-Example Spoken Term Detection task of the Search on Speech ALBAYZIN 2018 evaluation

Cabello¹,

Toledano²,

Tejedor³

2018

IberSPEECH 2018

Self Cite

View full text Add to dashboard Cite

Query-by-Example Spoken Term Detection is the task of detecting query occurrences within speech data (henceforth utterances). Our submission is based on a language-independent template matching approach. First, queries and utterances are represented as phonetic posteriorgrams computed for English language with the phoneme decoder developed by the Brno University of Technology. Next, the Subsequence Dynamic Time Warping algorithm with a modified Pearson correlation coefficient as cost measure is employed to hipothesize detections. Results on development data showed an ATWV=0.1774 with MAVIR data and an ATWV=0.0365 with RTVE data.

show abstract

ALBAYZIN Query-by-example Spoken Term Detection 2016 evaluation

Cited by 5 publications

References 51 publications

Search on speech from spoken queries: the Multi-domain International ALBAYZIN 2018 Query-by-Example Spoken Term Detection Evaluation

Search on speech from spoken queries: the Multi-domain International ALBAYZIN 2018 Query-by-Example Spoken Term Detection Evaluation

ALBAYZIN 2018 spoken term detection evaluation: a multi-domain international evaluation in Spanish

AUDIAS-CEU: A Language-independent approach for the Query-by-Example Spoken Term Detection task of the Search on Speech ALBAYZIN 2018 evaluation

Contact Info

Product

Resources

About