ASR-Free CNN-DTW Keyword Spotting Using Multilingual Bottleneck Features for Almost Zero-Resource Languages

Menon, Raghav; Kamper, Herman; Yılmaz, Emre; Quinn, John A.; Niesler, Thomas

doi:10.21437/sltu.2018-38

Cited by 9 publications

(6 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In QbE-STD task, the area under ROC curve (AUC) [18,19] is used to evaluate the accuracy of detection. The ROC curve is composed of false positive rate (FPR) and true positive rate (TPR) which are defined by:…”

Section: Evaluation Metricsmentioning

confidence: 99%

A Stage Match for Query-by-Example Spoken Term Detection Based On Structure Information of Query

Zhan

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

The state-of-the-art of query-by-example spoken term detection (QbE-STD) strategies are usually based on segmental dynamic time warping (S-DTW). However, the sliding window in S-DTW may separate signal of a word into different segments and produce many illegal candidates required to be compared with the query, which significantly reduce the accuracy and efficiency of detection. In this paper, we propose a stage match strategy based on the structure information of the query, represented with the unvoiced-voiced attribute of the portions in itself. The strategy first locates potential candidates with similar structure against the query in utterances, and further matches the query with Type-Location DTW (TL-DTW), which is a modified DTW with the constraints of pronunciation types and relative positions of paired frames in the voiced sub-segments. Experiments on AISHELL-1 Corpus showed that the proposed approach achieved a relative improvement of 30.51% in AUC against S-DTW and speeded up the retrieval.

show abstract

Section: Evaluation Metricsmentioning

confidence: 99%

A Stage Match for Query-by-Example Spoken Term Detection Based On Structure Information of Query

Zhan

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Word spotting traditionally is audio exemplar matching against spans of raw audio (Myers et al, 1980). It has been shown to be feasible in low resource scenarios using neural approaches (Menon et al, 2018b,a). Le Ferrand et al (2020 describes several plausible speech representations suited for low-resource word spotting.…”

Section: System Architecturementioning

confidence: 99%

Proceedings of the Second Workshop on Data Science with Human in the Loop: Language Advances

2021

View full text Add to dashboard Cite

Domain-specific conceptual bases use key concepts to capture domain scope and relevant information. Conceptual bases serve as a foundation for various downstream tasks, including ontology construction, information mapping, and analysis. However, building conceptual bases necessitates domain awareness and takes time. Wikipedia navigational templates offer multiple articles on the same/similar domain. It is possible to use the templates to recognize fundamental concepts that shape the domain. Earlier work in this domain used Wikipedia's structured and unstructured data to construct open-domain ontologies, domain terminologies, and knowledge bases. We present a novel method for leveraging navigational templates to create domain-specific fuzzy conceptual bases in this work. Our system generates knowledge graphs from the articles mentioned in the template, which we then process using Wikidata and machine learning algorithms. We filter important concepts using fuzzy logic on network metrics to create a crude conceptual base. Finally, the expert helps by refining the conceptual base. We demonstrate our system using an example of RNA virus antiviral drugs.

show abstract

“…A key disadvantage is that DTW is computationally very expensive and usually not feasible for large-scale continuous application (Menon et al, 2018b). Furthermore, for DTW the choice of input features has a greater impact on performance than, for example, for ASR, since in the latter case acoustic models can learn to adapt to different representations (Menon et al, 2018a).…”

Section: Introductionmentioning

confidence: 99%

“…In this way we take advantage of CNN-based searching, which is computationally efficient since it does not require alignment, to perform DTW-based matching, which requires a minimum of labelled data. We first proposed this approach, which we will refer to as CNN-DTW keyword spotting, in (Menon et al, 2018b) and later extended it in (Menon et al, 2018a) and in (Menon et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Feature learning for efficient ASR-free keyword spotting in low-resource languages

van der Westhuizen,

Kamper,

Menon

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We consider feature learning for efficient keyword spotting that can be applied in severely under-resourced settings. The objective is to support humanitarian relief programmes by the United Nations in parts of Africa in which almost no language resources are available. For rapid development in such a language, we rely on a small, easily-compiled set of isolated keywords. These keyword templates are applied to a large corpus of in-domain but untranscribed speech using dynamic time warping (DTW). The resulting DTW alignment scores are used to train a convolutional neural network (CNN) which is orders of magnitude more computationally efficient and suitable for real-time application. We optimise this neural network keyword spotter by identifying robust acoustic features in this almost zero-resource setting. First, we consider incorporating information from well-resourced but unrelated languages using a multilingual bottleneck feature (BNF) extractor. Next, we consider features extracted from an autoencoder (AE) trained on in-domain but untranscribed data. Finally, we consider correspondence autoencoder (CAE) features which are fine-tuned on the small set of in-domain labelled data. Experiments in South African English and Luganda, a lowresource language, show that BNF and CAE features achieve a 5% relative performance improvement over baseline MFCCs. However, using BNFs as

show abstract

ASR-Free CNN-DTW Keyword Spotting Using Multilingual Bottleneck Features for Almost Zero-Resource Languages

Cited by 9 publications

References 37 publications

A Stage Match for Query-by-Example Spoken Term Detection Based On Structure Information of Query

A Stage Match for Query-by-Example Spoken Term Detection Based On Structure Information of Query

Proceedings of the Second Workshop on Data Science with Human in the Loop: Language Advances

Feature learning for efficient ASR-free keyword spotting in low-resource languages

Contact Info

Product

Resources

About