Multilingual shifting deep bottleneck features for low-resource ASR

Nguyen, Quoc Son; Gehring, Jonas; Müller, Markus; Stüker, Sebastian; Waibel, Alex

doi:10.1109/icassp.2014.6854676

Cited by 17 publications

(12 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While MBN features have been shown to be useful in several speech recognition tasks (e.g. [20,21]), learned audio features face the same issue as word embeddings, as humans learn to extract useful features from the audio signal as a result of learning to understand language and not as a separate process. However, the MBN features can still be useful where system performance is more important than cognitive plausibility, for instance in a low resource setting.…”

Section: Introductionmentioning

confidence: 99%

Language Learning Using Speech to Image Retrieval

2019

View full text Add to dashboard Cite

Humans learn language by interaction with their environment and listening to other humans. It should also be possible for computational models to learn language directly from speech but so far most approaches require text. We improve on existing neural network approaches to create visually grounded embeddings for spoken utterances. Using a combination of a multi-layer GRU, importance sampling, cyclic learning rates, ensembling and vectorial self-attention our results show a remarkable increase in image-caption retrieval performance over previous work. Furthermore, we investigate which layers in the model learn to recognise words in the input. We find that deeper network layers are better at encoding word presence, although the final layer has slightly lower performance. This shows that our visually grounded sentence encoder learns to recognise words from the input even though it is not explicitly trained for word recognition.

show abstract

Section: Introductionmentioning

confidence: 99%

Language Learning Using Speech to Image Retrieval

2019

View full text Add to dashboard Cite

show abstract

“…[12] and thoroughly evaluated in [13] using five target lan guages and two language sets for multilingual training. Similar NN topology is used in [14], but the adaptation is not properly described and the evaluation is done on one language only.…”

Section: Introductionmentioning

confidence: 99%

Further investigation into multilingual training and adaptation of stacked bottle-neck neural network structure

Grézl

Egorova

Karafiát

2014

2014 IEEE Spoken Language Technology Workshop (SLT)

View full text Add to dashboard Cite

Multilingual training of neural networks for ASR is widely studied these days. It has been shown that languages with little training data can benefit largely fr om multilingual resources. We have evaluated possible ways of adaptation of multilingual stacked bottle-neck hi erarchy to target domain. This paper extends our latest work and fo cuses on the impact certain aspects have on the performance of an adapted neural network fe ature extractor. First, the performance of adapted multilingual networks preliminarily trained on different languages is studied. Next, the effect of different target unitsphonemes vs. triphone states -used for multilingual NN training is evaluated. Then the impact of an increasing number of languages

show abstract

“…Another way is to build bottleneck feature (BNF) extractor. The extracted target language BNFs are then used to train a Gaussian Mixture Model-Hidden Markov Model (GMM-HMM) or Deep Neural Network-Hidden Markov Model (DNN-HMM) recognizer [13], [14], [16], [19], [20], [23].…”

Section: Introductionmentioning

confidence: 99%

Submodular Based Unsupervised Data Selection

Zhang

2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYAutomatic speech recognition (ASR) and keyword search (KWS) have more and more found their way into our everyday lives, and their successes could boil down lots of factors. In these factors, large scale of speech data used for acoustic modeling is the key factor. However, it is difficult and time-consuming to acquire large scale of transcribed speech data for some languages, especially for low-resource languages. Thus, at low-resource condition, it becomes important with which transcribed data for acoustic modeling for improving the performance of ASR and KWS. In view of using acoustic data for acoustic modeling, there are two different ways. One is using the target language data, and another is using large scale of other source languages data for cross-lingual transfer. In this paper, we propose some approaches for efficient selecting acoustic data for acoustic modeling. For target language data, a submodular based unsupervised data selection approach is proposed. The submodular based unsupervised data selection could select more informative and representative utterances for manual transcription for acoustic modeling. For other source languages data, the high misclassified as target language based submodular multilingual data selection approach and knowledge based group multilingual data selection approach are proposed. When using selected multilingual data for multilingual deep neural network training for crosslingual transfer, it could improve the performance of ASR and KWS of target language. When comparing our proposed multilingual data selection approach with language identification based multilingual data selection approach, our proposed approach also obtains better effect. In this paper, we also analyze and compare the language factor and the acoustic factor influence on the performance of ASR and KWS. The influence of different scale of target language data on the performance of ASR and KWS at mono-lingual condition and cross-lingual condition are also compared and analyzed, and some significant conclusions can be concluded. key words: keyword spotting, submodular, multilingual data selection, language identification, recurrent neural network long short term memory

show abstract

Multilingual shifting deep bottleneck features for low-resource ASR

Cited by 17 publications

References 19 publications

Language Learning Using Speech to Image Retrieval

Language Learning Using Speech to Image Retrieval

Further investigation into multilingual training and adaptation of stacked bottle-neck neural network structure

Submodular Based Unsupervised Data Selection

Contact Info

Product

Resources

About