The Rwth Asr System for Ted-Lium Release 2: Improving Hybrid Hmm With Specaugment

Zhou, Wei; Michel, Wilfried; Irie, Kazuki; Kitza, Markus; Schlüter, Ralf; Ney, Hermann

doi:10.1109/icassp40776.2020.9053573

Cited by 42 publications

(27 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…E2E-Transformer (ID 9). We could not achieve any improvements on the DNN-HMM model (ID 2) using the SpecAugment despite trying different hyper-parameter tuning recommendations (Zhou et al, 2020). Overall, the best CER and WER results are achieved by the E2E-Transformer (ID 10) followed by the E2E-RNN (ID 6) and then the DNN-HMM (ID 1).…”

Section: Resultsmentioning

confidence: 98%

A Crowdsourced Open-Source Kazakh Speech Corpus and Initial Speech Recognition Baseline

Khassanov¹,

Mussakhojayeva²,

Mirzakhmetov³

et al. 2021

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume

View full text Add to dashboard Cite

We present an open-source speech corpus for the Kazakh language. The Kazakh speech corpus (KSC) contains around 332 hours of transcribed audio comprising over 153,000 utterances spoken by participants from different regions and age groups, as well as both genders. It was carefully inspected by native Kazakh speakers to ensure high quality. The KSC is the largest publicly available database developed to advance various Kazakh speech and language processing applications. In this paper, we first describe the data collection and preprocessing procedures followed by a description of the database specifications. We also share our experience and challenges faced during the database construction, which might benefit other researchers planning to build a speech corpus for a low-resource language. To demonstrate the reliability of the database, we performed preliminary speech recognition experiments. The experimental results imply that the quality of audio and transcripts is promising (2.8% character error rate and 8.7% word error rate on the test set). To enable experiment reproducibility and ease the corpus usage, we also released an ESPnet recipe for our speech recognition models.

show abstract

Section: Resultsmentioning

confidence: 98%

A Crowdsourced Open-Source Kazakh Speech Corpus and Initial Speech Recognition Baseline

Khassanov¹,

Mussakhojayeva²,

Mirzakhmetov³

et al. 2021

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume

View full text Add to dashboard Cite

show abstract

“…We also tried Specaugment, based on random on-the-fly time and feature masking used widely in seq2seq training [21]. All models are trained with the same masking parameters, following indications from [22].…”

Section: Training and Decodingmentioning

confidence: 99%

Improving Factored Hybrid HMM Acoustic Modeling without State Tying

Raissi¹,

Beck²,

Schlüter³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this work, we show that a factored hybrid hidden Markov model (FH-HMM) which is defined without any phonetic state-tying outperforms a state-of-the-art hybrid HMM. The factored hybrid HMM provides a link to transducer models in the way it models phonetic (label) context while preserving the strict separation of acoustic and language model of the hybrid HMM approach. Furthermore, we show that the factored hybrid model can be trained from scratch without using phonetic state-tying in any of the training steps. Our modeling approach enables triphone context while avoiding phonetic state-tying by a decomposition into locally normalized factored posteriors for monophones/HMM states in phoneme context. Experimental results are provided for Switchboard 300h and LibriSpeech. On the former task we also show that by avoiding the phonetic state-tying step, the factored hybrid can take better advantage of regularization techniques during training, compared to the standard hybrid HMM with phonetic state-tying based on classification and regression trees (CART).

show abstract

“…For SWBD, the Hub5'00 and Hub5'01 datasets are used as dev and test set, respectively. All the LMs used are the same as in [4] for TLv2 and [28] (sentence-wise) for SWBD. By default, word error rate (WER) results are obtained with full-sum decoding and a 4gram LM.…”

Section: Experiments 31 Setupmentioning

confidence: 99%

Phoneme Based Neural Transducer for Large Vocabulary Speech Recognition

Zhou

Berger

Schlüter

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

To join the advantages of classical and end-to-end approaches for speech recognition, we present a simple, novel and competitive approach for phoneme-based neural transducer modeling. Different alignment label topologies are compared and word-end-based phoneme label augmentation is proposed to improve performance. Utilizing the local dependency of phonemes, we adopt a simplified neural network structure and a straightforward integration with the external word-level language model to preserve the consistency of seq-to-seq modeling. We also present a simple, stable and efficient training procedure using frame-wise cross-entropy loss. A phonetic context size of one is shown to be sufficient for the best performance. A simplified scheduled sampling approach is applied for further improvement and different decoding approaches are briefly compared. The overall performance of our best model is comparable to state-of-the-art (SOTA) results for the TED-LIUM Release 2 and Switchboard corpora.

show abstract

The Rwth Asr System for Ted-Lium Release 2: Improving Hybrid Hmm With Specaugment

Cited by 42 publications

References 20 publications

A Crowdsourced Open-Source Kazakh Speech Corpus and Initial Speech Recognition Baseline

A Crowdsourced Open-Source Kazakh Speech Corpus and Initial Speech Recognition Baseline

Improving Factored Hybrid HMM Acoustic Modeling without State Tying

Phoneme Based Neural Transducer for Large Vocabulary Speech Recognition

Contact Info

Product

Resources

About